Linux

Asus Wifi Router AiMesh Odyssey

Send to Kindle

This post was rattling around in my head before the current stay-at-home situation arose. Somehow, it never found its way to my fingers, until now.

This will be a typical techie post by me, in that it will be long, rambling/meandering, and likely bore the people who used to mostly read my music blogs.

tl;dr

Most newer (even years old) Asus routers now support AiMesh, a method of turning normal routers into a mesh system. When configured correctly, it works remarkably well and has some serious advantages over pre-built mesh systems (like Google WiFi, Eero, etc.).

My start with Asus Routers

For years, I purchased cheap (often refurbished) routers. I typically flashed them with dd-wrt firmware to make them better than new.

On December 13th, 2013 I broke the mold. I purchased my first high-end router, the Asus RT-AC66U. It was my first “AC” router (now called WiFi 5) and the first time I ever spent that much money on a router of any kind ($179.99 before tax!).

It has a (theoretical and nonsensical) top speed rating of 1750 megabits per second (mbps). That’s still pretty good even by today’s standards.

Back then, this router was glorious. I was very happy with my purchase and it was my main router for many years. It only got replaced when I switched to Verizon FiOS which at the time came with their own branded router.

This router still gets firmware updates (what a credit to Asus!). More on that later in this (happy ending) saga.

The big move

In 2015 we moved from NY to VA. Our house in VA is large, but not that large. The layout puts our master bedroom far away from every other room (no centrally placed router would reach that room).

Mesh routers hadn’t exploded in popularity yet, so I never considered one at the time. Given that I was installing FiOS there (the previous owner had FiOS as well), I knew that my main router would be a Verizon branded one.

I had an electrician put in a hard-wired Ethernet port in three rooms, in addition to the closet where the FiOS router lives. There is an Ethernet port in the wall in our office, the master bedroom, and the basement (where the TV is). Each of the three rooms is a home-run back to the closet, so each of the rooms connects into one of the four LAN ports on the back of the FiOS router.

The office is very close to the closet with the main router in it, so to begin with, I didn’t put another router in there.

The master bedroom and basement each got routers. Those routers were put in AP mode (Access Point, or Bridged). That meant that the main FiOS router still handed out all IP addresses (no matter the room or which WiFi router our devices were connected to).

For simplicity, each of the AP routers got a separate SSID (the WiFi Network Name). Devices that were fixed in a particular room always connected to their local SSID/Router (e.g., the TV, or a Fire Stick/Roku, etc.). Our cellphones had to switch to a new SSID when we moved from one room to another.

Since the main router signal really doesn’t reach the master bedroom at all, the phones would switch pretty quickly (without any manual intervention), simply because they would lose the signal completely and immediately begin searching for a new signal.

It wasn’t instantaneous, so you would have to think about starting something before you walked out of a room, because the connection would definitely drop for a few seconds.

After a few months, the FiOS router started getting flaky on the AC channel (this eventually straightened out years later after some arbitrary firmware update). This led me to pull out my old Asus RT-AC66U router (which was in a drawer for a while by then) and install it in the office.

Even though the office was close to the main router, I could no longer trust the WiFi on the main router, so I used the wired connection in the office to hook up the Asus as an AP and all was well again.

Growing the Asus family

The above setup continued for a while and worked fine. In the master bedroom I had an old TP-Link router (another brand that I really like). It worked fine, but wasn’t the fastest thing and would rarely cause me tiny frustrations.

Six weeks before buying my first Asus router, I bought a Nexus 5 phone from Google. That marked my switch from Verizon (where I was using the Galaxy Nexus) to T-Mobile. I’ve been a happy T-Mobile customer ever since.

My house had effectively zero coverage on T-Mobile when we moved in (it’s marginally better now, though I rarely ever have to use it in the house). After a few months in the house, I saw an ad for a T-Mobile Cellspot. It’s a rebranded Asus RT-AC68U router (one step up from the 66U that I owned).

T-Mobile was selling them very cheaply (much cheaper than the Asus branded 68U). I bought it not only because the price was excellent (if memory serves me well, I think I paid $75 for a new one, when the 68U was going for $199 new), but also because it was (supposedly) optimized for T-Mobile WiFi calling (which given my cell signal at home sounded like a great addition).

I replaced my (rarely) flaky TP-Link router with a T-Mobile Cellspot one in the master bedroom.

The dreaded, complicated upgrade

The new router ran great for months, but I never used the WiFi calling feature (that’s a long story for another post that I will likely never write).

At some point I stumbled on a post about “Turning a T-Mobile Cellspot into a Full Blown Asus RT-AC68U“. If you skim that article, and more importantly the tons of comments with a lot detail in them, you’ll see that this is not for the feint of heart.

Fortunately or otherwise, this is exactly the type of thing I enjoy (hey, you get your kicks your way, I’ll get mine my way…).

With some twists and turns, and needing to read a bunch of comments when I got stuck, I eventually turned the Cellspot into a full blown Asus router.

Why bother?

That’s a fair question. The Cellspot worked perfectly (for some definition of perfect). There are a few good reasons to consider the upgrade. By far the most important one is that the Cellspot firmware was way behind, and while occasionally upgraded, pretty rarely.

That meant that when the various kracks that have been discovered in recent years against WiFi routers are revealed, a real Asus router will be patched months (or years!) before the Cellspot will. That’s reason enough to bother if you don’t require any special handling of the WiFi calling feature for T-Mobile specifically.

Another reason to upgrade is to flash different firmware, e.g., dd-wrt (mentioned above). That’s not possible from the base Cellspot firmware, but is from the Asus firmware.

Finally, if you want to run the AiMesh software, you need to be on the real Asus firmware.

A quick look ahead

I’ll deal with this later on, but there are newer (shorter, better) ways of upgrading a Cellspot, and very important warnings and caveats which didn’t exist at the time I upgraded my first one.

The point of this interlude is to tell you to read on rather than follow the instructions in the article linked above.

Another Asus added to the family…

A year later, my better TP-Link router running in the basement started to have some issues (again, I think it was a firmware issue that I later resolved). I decided to replace it with another upgraded Cellspot.

I bought one refurbished on Amazon for $48. I followed the same upgrade instructions linked above, and had another working Asus RT-AC68U router installed in the basement the same day it arrived.

I now had four routers in the house. The main FiOS one which was mostly acting as a wired router to the Internet (a few legacy devices, security cameras, etc., were connected directly to the 2.4Ghz channel on that router), plus the original RT-AC66U in the office, and the two upgraded Cellspots, in the master bedroom and basement.

What? Asus doesn’t make infinitely perfect hardware?

About six months ago, I walked into the office and the old 66U router was dead. No lights, no Internet (obviously).

I disconnected the cables and pulled out the TP-Link Archer C9 that had previously been running in the basement. That’s the one that I asserted was flaky only because of a specific firmware.

I reconfigured it to take the place of the old 66U, made sure it was current on firmware, and turned it on. Problem solved, we were back in business.

I decided to try and diagnose what went wrong with the old 66U (just out of curiosity, as it was 6 years old at the time and didn’t need to provide any additional service to make it one of the more outstanding tech purchases).

I connected it directly to my laptop using the wired port and fired it up. The lights blinked for a second and then went dead. It only took me one more try to realize what was wrong. The power button was broken. It simply wouldn’t click and stay on.

In typical MacGyver mode, I found a round hard piece of plastic, scotch taped it on to the power button, then put a rubber band around that to ease the pressure on the scotch tape.

Voila! A working 66U router, once again…

I swapped it back for the TP-Link, which was now perfectly configured to be an instant backup router should my MacGyver skills prove unworthy.

Wasn’t this supposed to be about AiMesh???

Oh yeah, though I did specifically mention that this would meander and ramble and I didn’t want to disappoint on that front either…

Unfortunately, a bit more meandering is necessary, just for historical accuracy, not to discuss the merits of AiMesh.

Getting into trouble

Before getting the second Cellspot router, I upgraded the first one using the built-in Asus firmware upgrade tool once, and it worked great.

When I got the second one, I (of course) upgraded it to the same version of firmware as the first one was on, with no issues.

Toward the end of last year, another krack was discovered, and I checked whether Asus had an updated firmware to mitigate it. They did.

I updated the old 66U first, and it upgraded perfectly.

I updated the first 68U and it reverted back to the original Cellspot firmware (which had even more issues than I was currently trying to fix!).

Whoah, what just happened?

A bit of Googling and I found that Asus decided that if they noticed that a Cellspot router was being flashed with Asus firmware (rather than a T-Mobile branded firmware), they would roll it back to the original.

Darn!

Silver Lining?

This caused me to find newer methods of turning it back into an Asus router, including ways to thwart Asus from rolling it back. The old method (linked above) still works, and has the appropriate warnings and methods to avoid the rollback, but it’s still more complicated than the new ones.

This is a link to the instructions that I used the second time around. It looks long and complicated, but that’s because there are three different (analogous) methods for accomplishing the upgrade. The key point is that this avoids the full downgrade which the original method requires.

When I did this and got my first router back to the new firmware, and made sure that it wouldn’t downgrade again, I flashed the second one as well, apparently successfully.

Apparently???

Wait, what? Either it worked or it didn’t. Well, yes (and no).

I was so smart (being a seasoned techie) that I was incredibly stupid (being a seasoned know-it-all).

The first time that I upgraded each router, I meticulously followed the dozens of steps in the original article linked above. Amazingly, following detailed instructions worked (can you believe it?).

This second time around, using the simplified instructions (which are 100% accurate and would work if you followed them exactly!), I skipped one crucial section (a few commands) because I assumed that they were unnecessary the second time around (meaning, I thought I had the correct files from the first time around just sitting in my folder waiting to be reused).

Again, why apparently then? Because I don’t end up using the router in the basement all that often, and it took trying to get AiMesh to work (still coming, I promise) to finally see what I had done so wrong…

AiMesh, finally!

Well, actually, getting closer, not there quite yet…

Now that everything was running fine (or so I thought), I decided to finally experiment with turning on AiMesh in all three Asus routers.

I really didn’t need it, my setup was working well enough, but I was curious and now that I was running the latest version of Asus firmware on all three routers, I was in a position to find out. I could always roll back to non-AiMesh mode if it wasn’t to my liking.

Unfortunately, I hit a snag immediately. It turns out that the old 66U is not capable of running AiMesh software. There is a newer revision (RT-AC66U rev B) that can run full AiMesh, but mine is too old and can’t do it.

So, I popped on to Amazon and ordered another Cellspot for $48 (this time, it was labeled renewed rather then refurbished).

Unfortunately, I compounded my error by skipping the exact same block of steps on this newest router as I had on my others, because I hadn’t yet noticed the problem that I had introduced into my network…

Turning on AiMesh, finally, really, this time

I flashed the real Asus firmware onto the newest Cellspot and retired the old 66U once again. I was now ready to flip the switch and turn the three routers into an AiMesh mesh network.

All attempts to get them talking to each other failed! After some searching, it seemed that some people had more success using the Asus Router App on their phones, than using the web browser interface.

I broke down and installed the app on my phone. I did seem to get a bit further, but if I got something going on one router, another would disappear, and then that would be flipped. It was maddening.

Discovering the problem

Well, the problem was entirely created by me, so I was the problem. The crucial steps involved the following:

Upload original_cfe.bin to https://cfeditor.pipeline.sh/
Select 1.0.2.0 US 1.0.2.5 US for AC68P or 1.0.2.0 US AiMesh for AC68U with AiMesh as Source CFE
Download the new .bin
rename it to new_cfe.bin

I assumed that having done that once on the original router, I had the correctly modified CFE (now called new_cfe.bin). Meaning, I thought that all Asus routers (at least of the same model number, which mine were), shared the same identical new cfe.bin file.

You’ve all heard what the definition of the word assume is, right? I’ll spare the gentler ears/eyes from seeing it here again…

It turns out that the file is unique to each and every router. Why? Because among other things, it contains the MAC Address of the routers ports (both Ethernet and WiFi) embedded in it. So, by reusing the same cfe.bin file on all three routers, they were all running with the same exact MAC Address.

To be clear, they each had different IP addresses assigned, but that doesn’t make the problem better. The way local networking works, there is an ARP table maintained that tells the network how to reach the physical machine associated with an IP address, by translating it into the MAC Address.

So, when I tried to reach any of the three routers via their IP address, all of them returned (at a very low level) the same MAC Address, and therefore it was entirely random (perhaps based on distance in the house) as to which router would see my request!

Ugh.

The solution

Once I understood the problem, the solution was obvious and straightforward, but by no means simple. I needed to fix the individual cfe.bin files, but I could no longer follow the original instructions (uploading them to a website which would edit them) because I didn’t have the original files to upload!

Worse, I needed to figure out which MAC Address was correct for which router, which meant going to each of them and finding the stickers with the serial numbers and MAC Address printed on them.

Once I did that, I had to use a HEX Editor to load up each file, find the wrong MAC Addresses (yes, plural, since there are multiple interfaces in each router) and type over them (very carefully).

Then I needed to copy them over to the correct router, flash them, reboot the router, and pray.

Yes, that worked!

Are we there yet?

So, was I really done? Unfortunately, not quite.

I was able to get AiMesh going, but the speed in the bedroom was pathetic (reliable, but pathetic). The speed in the basement was great!

This one didn’t take long to diagnose, but it did take a while to fix…

By default, AiMesh sets the backhaul (how the mesh routers communicate with each other, rather than with the client devices or the Internet) to auto.

In the case of the basement router, that ended up using the wired connection over the Ethernet cable (which is exactly what I expected the default to be).

In the master bedroom, even though the router is fully wired like the basement one is, auto defaulted to wireless backhaul.

If you recall from a few days ago, when you started reading this post, the master bedroom is too far away to get a reliable signal, so the backhaul was awful (amazing that it worked at all!).

The solution is simple, force the backhaul to be wired. Yes, simple, in theory, but I couldn’t find any way to do that!

More searching on the Internet and I finally found a single forum post where someone linked to the official guide with highlighted screenshots.

Bless that individual, and Google, for surfacing the correct post (after much tribulation).

Here is a link to the guide, with step 6 being the secret sauce to finally see where the default backhaul could be changed/

Conclusion

So, was it all worth it? Yes, of course.

First, I love technology puzzles, even ones created by me. Once I screwed up the settings really badly, I just had to figure out how to get myself out of it. It wasn’t fun (on any level), but it was instructive, informative, and satisfying (in the end).

Much more importantly, I am now running a full mesh network and I like it. Our phones don’t drop when walking from our office to our bedroom. All three routers are effectively managed from the one main AiMesh one.

Why AiMesh is really cool

Most importantly, it’s a mix and match network. You don’t have to buy kits. You don’t have to have identical routers at each node. As long as a router supports the AiMesh firmware (which many Asus routers do!), it can be a node (or the master) of your AiMesh network.

This is crucial. Today, I don’t own a single WiFi 6 (AX) device. So, it would be overkill for me to buy a WiFi 6 router, let alone a WiFi 6 Mesh Kit.

However, if/when I get a new laptop (I’m typing this on a 6-year-old one) that has WiFi 6 in it, or a new phone (mine is 2.5 years old), I’ll be able to get an Asus WiFi 6 router (any of them!) and use it as my main AiMesh node (and place it wherever I use the laptop most frequently, which now, is in the office).

I won’t have to change the other routers, or change any settings on the other routers either. They will all just work. My laptop (and phone) will work with WiFi 6 when they’re connected to the new router, and automatically and gracefully downgrade to WiFi 5 when they roam to another AiMesh node that’s still on WiFi 5.

Further, I can even do the WiFi 6 upgrade piecemeal. For example, I could get a lower-end WiFi 6 AiMesh router first, and make that the master. Then, as I have more devices that can take advantage, I can get a higher-end WiFi 6 router, make that the main one, and move the older WiFi 6 router into the bedroom.

The ultimate beauty is that each of the routers can always be instantly returned to be non-AiMesh routers. So, I can pass them on to friends when I replace them with a WiFi 6 one and those people can use them as standalone routers, AP bridges, or create or augment an AiMesh of their own.

That’s what makes these more flexible than full-time mesh systems.

Also, it doesn’t hurt that the models I’m running can be picked up for $48, if you’re willing to be super careful and avoid the stupid mistake that I made.

Installing CrashPlan on a PogoPlug Pro

Send to Kindle

There are critical updates embedded in this post, added on 9/3/2011, all preceded with Update:. You can apply those instructions separately if you’ve already completed the rest of this installation. Updates will be marked with End Update to allow for multi-paragraph updates. I will strike-through the parts that were replaced, so that you can safely ignore them if you’re going through this guide for the first time.

PogoPlug Pro is an amazing device (coupled with an amazing service). CrashPlan is an amazing piece of software (and also provides a fee-based amazing service). I’ve had both for a while and think very highly of them.

To solve a number of my own problems (not caused by either service!), I decided to investigate marrying them (the PogoPlug device, with the CrashPlan software). To be slightly more accurate, I wanted the device to perform an additional function (beyond backups). I wanted it to be my primary DLNA server.

Caution: none of what follows is supported by either company. You will be voiding your PogoPlug warranty and CrashPlan does not support this configuration of their Linux software. Proceed at your own risk!

My primary reason for installing CrashPlan on this device is to compensate for the pathetic upload speed provided by Time Warner, all of 485Kbps, shared with my wife, for normal Internet use, VoIP calls and backups. In other words, not really a useful real-time backup solution. Since we are often at other, very high-speed locations, I still believe that paying for the CrashPlan online backup service is the way to go (and gives me great comfort), but when I’m home, I wanted a local solution (that didn’t involve plugging in a hard drive to my laptop).

Since I was able to do this, successfully, the instructions can be found on the Net. Since it took me way longer to find the various pieces, let alone get them to work, than I felt it should have, I’m writing this (for myself, as well as for others who might give up more easily than I did). None of the credit goes to me, I’m just collecting the wisdom of others in one place, hopefully an easy one to find.

Update: All sections marked Update: that apply to Java and udev were courtesy of Ankit Mohan. Ankit used this guide to get CrashPlan running on his PogoPlug, then dug in a lot more than I did to solve the problems I described. I am indebted to him. End Update.

There are a number of Linux distributions available for the PogoPlug Pro (an ARM-based device). I chose Arch Linux because it was the most prominent one I found and because it has a very good reputation independent of the PogoPlug. The specific ARM implementation has it’s own site, which is where I started my odyssey.

After reading the overview on that page, I clicked on the Installation tab. The instructions there are extremely clear. The first time I followed them, the formatting of my external drive failed. I ended up formatting the drive while it was connected to a laptop running Linux, but all of the other instructions on that page worked. I will repeat them here, so that anyone who doesn’t want to link off of this page can simply follow all the way through.

You have to register your PogoPlug at my.pogoplug.com. This is required, even though we will shortly be disabling the service, since this is the only way to get SSH access to the device. You will be able to reverse the process, returning the device to a full PogoPlug in the future, should you desire that, but it’s not a dual-boot situation where you decide which version you want it to be.

Once you’ve enabled SSH on the device, you can set your own password. The username will be root. The default password for a PogoPlug Pro is ceadmin (as noted, you can change it via the website, or once you log in, with the passwd command. Change it!

One of the steps that they don’t cover is discovering the IP address of your PogoPlug, so that you can SSH to it. In a home environment, this is relatively easy (for the geeks among us). You login to your router, look at the list of attached devices and easily spot it.

I’m installing the software on a second device as I type along. I’m in an office environment and don’t have access to the router. There are hundreds of devices in the office. I had to write down the MAC address of the PogoPlug, go over to a system administrator and ask him to search the DHCP log files for that MAC address. He did and I found out the address that was assigned to it. Whew.

I successfully logged into the device, just to make sure it worked, while it was still an official PogoPlug. That step was optional, but comforting. Since the next step is to power down the device (which you can safely do by just pulling the power cord, especially if you have no hard drives attached yet), since I’m logged in as root, I typed /sbin/halt instead, to be a little safer. Wait 60 seconds (for added safety), then pull the power cord.

We’re going to install Arch Linux on an external drive. The only thing that will be changed on the PogoPlug itself is the boot sector, which will now point to the external drive (that’s what would need to be reversed to restore the PogoPlug functionality).

With the PogoPlug powered down, attach only the drive that you intend to install Arch Linux on. This way there will be no confusion or errors. Later on, if you want multiple drives attached (for backups and/or media files) it will be easy to add them. I am using a 2TB Fantom Drives to do my install.

Once the drive is attached (and turned on), plug the power cord back into the PogoPlug and wait for it to boot. Then SSH back on to it (the root password will be what you set it to, or ceadmin if you didn’t change it). The box is still running the PogoPlug software, with your drive attached to it.

Type: killall hbwd

That will stop the PogoPlug software from running on the box. We don’t want it to interfere with the installation of Arch Linux. You might have to wait a bit for the service to stop. If you want to check, type the following:

ps | grep hb

The only result should your grep process. Then, you can type:

/sbin/fdisk –l

The last line of output should start with /dev/sda1. That means your disk drive was found and has a partition on it (it’s likely formatted already). We are about to erase everything on the disk, so be absolutely sure that you want to continue with this adventure before doing that! If you’re ready, type:

/sbin/fdisk /dev/sda

That will bring up the fdisk program on the entire drive (sda as opposed to sda1 which is the first partition). You will now have a prompt that is directly from the fdisk program. We will type a number of one character commands. Right after you type the character and press enter, fdisk will go off and do what you asked it to.

Type: o

That will clear the partition table so that the disk will become unusable (for the moment). As you can see from the messages, nothing has been committed in stone as yet (very soon). This has modified an in-memory copy of the partition table.

Type: p (to verify the above, that there are now no partitions)

Type: n (press Enter, this will create a new partition)

Type: p (to make it a Primary partition. At this point, I’ll stop saying “Press Enter”, but you still have to!)

Type: 1 (to make this partition #1)

At the next two prompts (First and Last cylinders), just Press Enter to accept the defaults (you are making the entire disk available as the first and only partition).

Now comes the destructive part. This will actually wipe out any data you had on the disk (but still doesn’t modify the PogoPlug in any way!).

Type: w (this writes the partition table back out to the disk)

You are now back at the command line. If you’re a paranoid type (or just careful), you can verify that things worked by repeating the fdisk command and listing out the partition table, all in one shot:

/sbin/fdisk –l

This is the output on my system:

Device Boot      Start         End      Blocks  Id System
/dev/sda1               1      243201  1953512001  83 Linux

2TB, in one partition, marked for use by a Linux system. Now we need to actually create the filesystem, which in our case will be an ext3 one. This will require downloading some commands that will need to be executed. Here are the steps:

Type: sync (to flush any filesystem changes)

Type: cd /tmp (to change to a temporary, writable working directory)

Type: wget http://archlinuxarm.org/os/pogoplug/mke2fs (to retrieve the program mke2fs)

Type: chmod 755 mke2fs (to make it executable)

Type: ./mke2fs -j /dev/sda1 (the leading dot is critical. This will format the partition to an ext3 filesystem)

The above command can take quite a while, depending on the size of your external drive. This is the command that failed for me on my first PogoPlug, so I ended up having to detach the drive, connect it to a Linux laptop, perform the same exact command as above (which was already available, I didn’t need the wget part) and then reattach the drive to the PogoPlug.

Note: it failed for me again. I was able to format it using the built-in /sbin/mkfs.ext2 command (passing in the “-j” flag), but I didn’t trust that it was building a true ext3 filesystem (ext2 + journal). So, I disconnected the drive from the PogoPlug, attached it to a VirtualBox VM on my Win7 laptop, and formatted it there as a real ext3. Took forever, but it worked.

Whether the mke2fs command worked for you or whether you had to format the drive separately, like I had to on two separate installations, you’re now ready to install Arch Linux on the external drive. You should already (still) be in /tmp, but to make sure, feel free to type: cd /tmp

Type: wget http://archlinuxarm.org/os/oxnas/oxnas-install.sh (that retrieves the install script)

Type: chmod 755 oxnas-install.sh (this makes the script executable)

Type: ./oxnas-install.sh (this starts the script, which will send lots of messages to your terminal window. It also downloads the root filesystem image, which is roughly 129MB, so it can take a while if you don’t have a fast network connection.)

When it’s finally done (took between 5-10 minutes on a very high speed connection in the office), the output should look like this if it succeeded (at the very end):

#############################
## Looks good!
# Sync …
# Unmount
# Reboot to enter into Arch Linux ARM

Note the looks good! and then the instructions to reboot. That’s what we’re going to do next.

Type: /sbin/reboot (cross fingers!) Winking smile

This will immediately disconnect you from the terminal window you were in. You need to wait a few minutes for the orderly shutdown of the PogoPlug, followed by the booting up of Arch Linux ARM. You can watch the lights on your external drive to see when there is activity on it, indicating that the booting has begun. When the light settles down, the boot is complete.

We’re ready to log back in (this time to the new operating system), and the password has been changed to a new default one. The user is still root, but now so is the password (root), which you should change right away with the passwd command. It’s quite possible that the IP address of the box has changed during the reboot, so please verify the new (or existing) one, before SSH’ing back on.

If the IP address did not change, then you might have to remove the old key associated with it, or ssh might refuse to connect, thinking it’s a security violation. If you get the same IP address again, you may need to run the following command first (on your local machine, the one you are SSH’ing from):

Type: ssh-keygen -R 192.168.1.123 # (using your device’s IP, which won’t likely be 192.168.1.123)

The following First Steps page on the Arch Linux ARM Wiki explains the above, and gives you a number of other useful tips. I followed them religiously the first time through, but I changed the order a bit this time around and it saved a bit of typing (or I think it did).

Instead of going through the above, this time I updated all of the packages right away. I believe that it installed openntpd and updated the /etc/rc.conf file (one of the first steps that I performed manually the last time). You can do what I did, then check if openntpd is installed and running.

Type: pacman –Scc (clear out old packages. I said YES to the first, and NO to remove unused repositories)

Type: pacman –Syu (this will do a large update, first syncing the repositories, then updating all packages)

Now comes a crazy part. I say crazy because by the time you read this, perhaps the maintainers of Arch Linux ARM will have updated the repositories and this will no longer be necessary. Then people will think I’m an idiot, so be it, I’m putting it in here because it can’t hurt!

Type: pacman –Sy udev-compat (to fix a problem with udev + syslog-ng taking up 100% of your CPU)

The 100% cpu problem might be happening as you read this (if you’ve done the previous steps already). It might be filling up your disk in /var/log as well. We can check that in a minute (here’s the thread that helped me: there’s a typo in that thread, “sleep3” should be “sleep 3”), but first, let’s do a few more things and then reboot.

Update: Type pacman -Sfy udev-oxnas udev-automount (this fixes the udev problem noted above, now struck-through. I added the f option to pacman to force the removal of the bad udev, or udev-compat that you installed if you’ve already completed these instructions. You will have to say Y to the prompt to remove udev, which defaults to N.) End Update.

Let’s create a swapfile:

Type: dd if=/dev/zero of=/swapfile.img bs=1M count=1024 #for a 512MB swapfile, use count=512

Type: mkswap /swapfile.img (to turn the file we just created into a valid swapfile)

Type: swapon /swapfile.img (to see whether you get any errors, you shouldn’t)

Now we’ll edit /etc/rc.local (use your favorite editor, I use “vi”, you might prefer “nano”) to add exactly four one lines after the comments:

swapon /swapfile.img
kill $(pidof udevd)
sleep 3
udevd &

The first line turns the swap on after each reboot. The next three lines kill the bad udev (even after updating the udev-compat the process sometimes misbehaves at boot), then sleeps for three seconds and restarts udev, which makes it seem to behave correctly so far.

OK, time to reboot and see if we have a stable system:

Type: sync (flushes the memory to disk)

Type: reboot (you should lose your ssh connection right away)

Wait until the disk activity settles down a bit, then ssh back in.

Type: top (to see what processes are running. If udev and/or syslog-ng are at the top, something isn’t good)

Type: q (to exit top, whenever you’re done looking around)

You can follow any of the additional instructions for setting up a user, adding sudo, changing TimeZone settings, etc. All are linked from the First Steps above.

Since the reason I did this was to install DLNA, I’ll cover that first (it’s really short), then we’ll move on to the heavier CrashPlan setup. Skip the next few lines if you have no interest in DLNA.

Type: pacman -Sy minidlna jack (this names two packages, but will install something closer to 44, with dependencies)

You should edit the file /etc/minidlna.conf and change any variables (where the files are stored, where to store the DB, what you want to call your DLNA server, etc.). You can read the Wiki page (linked above) to see the more important entries.

Then add the word minidlna at the end of the DAEMONS= line, which should be at the bottom of the /etc/rc.conf file. This will auto-start the DLNA server every time the PogoPlug is booted. To start it right now, type: /etc/rc.d/minidlna start

Update: I discovered two things. 1) Many devices, e.g. Google TV and Sony Bravia TV, don’t show any of the files with a filetype of FLV. It turns out that if you simply rename the filetype to AVI (probably others), the files play fine (assuming they were encoded with H.264). 2) After you populate your media directory, run the following command and wait patiently: minidlna -R to force a build of the database. You probably want to kill all minidlna processes when this is done and start it again (minidlna alone on the command line is good enough). You can tail the minidlna.log file (in your database directory) to know when the database rebuild is done. End Update.

Whew. Finally ready for the very tricky and long installation of CrashPlan. This is not for the faint of heart, nor is it in any way supported by CrashPlan. It works for me (and obviously others), but you’ll have to be the judge as to whether the hassle is worth it for you.

Let’s start with crediting the place that got me unstuck, the CrashPlan support forums! Kudos to CrashPlan for allowing this type of discussion on their forums, even though they don’t support this configuration. Here’s the thread, but all of the interesting bits are in the long comment by Torbjorn. It was really hard for me to find, because I was searching for the word PogoPlug. This solved the problem for Sheeva (the baseline of the Pogo), but it’s not quite identical.

Type: pacman –Sy openjdk6 cpio (we need to get a working Java installed and CrashPlan will require the cpio package separately)

The next step (according to Torbjorn) is to download an ancient (2005) libjtux source package, apply a patch and compile it. He supplies a pointer to the source (amazingly, still available) and the patch file is available as an attachment to his comment. You can grab both from the the thread linked above. If you do, you will likely have to download a bunch of development packages (using pacman), starting with gcc.

Instead, I will attach the completed libjtux.so that I built (following those instructions), to save you time, effort and potential errors. I also just grabbed it from my first build and applied it to my second, for the same reasons.

Now we need to install CrashPlan itself by heading to the download page for Linux. I right-clicked on the download button (currently version 3.0.3, but the software auto-updates after the first install). I copied the link location. Back on the PogoPlug:

Type: wget http://download.crashplan.com/installs/linux/install/CrashPlan/CrashPlan_3.0.3_Linux.tgz (that should work, but starting at the http part, just paste in the link you copied if it’s newer than 3.0.3)

Type: cd /tmp

Type: tar –xzf WHEREVER_YOU_DOWNLOADED_THE_CRASHPLAN_FILE (which could be /tmp to simplify matters)

Type: cd CrashPlan-install

Type: ./install.sh (all of the defaults seem reasonable to me, though I did put my archives in another directory. You will need to page through the license file with the space bar and accept that as well. The init scripts on Arch are in /etc/rc.d, which is the other thing I changed from the default.)

When this is done, it will report that it has successfully started the CrashPlan service. It did not. That’s because we haven’t yet replaced the libjtux.so that comes with CrashPlan. The problem is that it was compiled with an Intel i386 architecture.

Type: cd /usr/local/crashplan

Type: mv libjtux.so libjtux.so-ORIGINAL (no real need to save it, other than to memorialize the changes we’re making)

Type: cp WHERE_YOU_DOWNLOADED/libjtux.so . (this copies my version from wherever you downloaded it, or you can right click my link above, and wget directly from my website to this directory)

Torjborn mentions editing a file to add jna.jar to it. I didn’t need to do that, and the directory he references doesn’t exist. I think it’s from a different installation of Java (for the original Sheeva) and not necessary when using the openjdk6 package.

Update: This next part solves the timing delays, apparently among a number of other issues that I wasn’t even aware of! Once again, thanks to Ankit for figuring this all out.

You will need the nss package installed. Mine was there after the major update above. If you don’t have it installed, type: pacman -Sy nss

The file that’s missing from openjdk, which is causing all of the problems with CrashPlan, is libjnidispatch.so. It’s part of the jna package (Java Native Access). You’ll have to download a Debian package and extract the file.

Select a location near you from this link: http://packages.debian.org/sid/armel/libjna-java/download. In the US, I chose this link directly: http://ftp.us.debian.org/debian/pool/main/libj/libjna-java/libjna-java_3.2.7-4_armel.deb

Assuming you downloaded that file to /tmp, here are the commands to extract the file we’re interested in:

Type: cd /tmp

Type: ar vx libjna-java_3.2.7-4_armel.deb (this will extract three files, one of which contains the file we’re interested in)

Type: tar -xzf data.tar.gz (this will unpack the tar file, creating a series of directories in /tmp/usr)

First we need to create the target directory for this. Since mkdirs (make nested directories) doesn’t exist by default, we’ll execute a number of consecutive mkdir commands. Hint: you can hit up-arrow after each command and just append the next directory name in the series of mkdir commands.

Type: cd /usr/local/crashplan/lib

Type: mkdir com

Type: mkdir com/sun

Type: mkdir com/sun/jna

Type: mkdir com/sun/jna/linux-arm

Type: cp /tmp/usr/lib/jni/libjnidispatch.so com/sun/jna/linux-arm/ (now we have the file in the correct directory)

Type: cp -p jna-3.2.5.jar jna-3.2.5.jar-ORIGINAL (this isn’t strictly necessary. Aside from documenting our change, it allows us to recover from errors more easily)

Type: jar uf jna-3.2.5.jar com/sun/jna/linux-arm/libjnidispatch.so (this is the critical step, inserting the library into the jar file that CrashPlan uses. I’m not sure the com/sun/jna/linux-arm directory creation is necessary, but better safe than sorry)

It seems that in addition to solving the timing issues (long delays in starting up), this actually dispatches (duh) maintenance tasks like pruning, synchronizing, etc. It’s all good, but there is a side-effect (well worth it!) that the Java process now takes up a significant amount of CPU, even when backups aren’t active, since it appears to actively maintain the system. Previously, Java would consume 0% of the cpu when it wasn’t backing up.

End Update.

We’re ready to start CrashPlan (at least to see if it comes up and stays up). We can do it by hand, but I am going to add it to the DAEMONS list at the end of /etc/rc.conf (like we did with minidlna) and reboot the machine to ensure that it comes up on it’s own. The daemon name to add right after minidlna is crashplan (lower case).

Type: /sbin/reboot

If you did everything above correctly, when you log back in, there should be a java process running, with CrashPlan as the application. It can a few minutes to completely initialize. In that case, we’re done, right? Wrong! CrashPlan is indeed sitting there, waiting to go to work, but it needs to be configured to allow your machine(s) to start backing up to it. Under normal circumstances, this would be trivial to do, but since the PogoPlug is a headless server, we have to jump through a few final hoops.

Once again, CrashPlan support to the rescue (again, for a completely unsupported feature). If you want to understand the details, or are enough of a techy to prefer the theory, I urge you to just read the CrashPlan document and skip to the final section of this post. If you want fewer steps (and warnings) to follow, I’ll give you the bare necessary steps here.

They key is that on every machine that has CrashPlan installed, there are two programs: 1) the server that does all the work and 2) the GUI (graphical user interface) that connects to the server when launched, and allows you to configure and monitor the server process. On the PogoPlug, we only have #1. The good news is that the GUI speaks to the server over the network. By default, that network is local to the machine that the server is running on, but with some ssh magic (and a little editing of a configuration file), we can make that a remote connection.

All of the work is going to be done on your desktop or laptop, where you already (presumably) have CrashPlan running. This is likely the machine that you want to backup to the PogoPlug. Let’s just call it laptop so that it’s obvious it isn’t the PogoPlug.

On laptop, make sure that the CrashPlan GUI isn’t running. If it is, exit the application. Find the conf directory associated with CrashPlan on your system. On my Windows 7 x64 machine, it’s this directory: “c:\Program Files\CrashPlan\conf”. In that directory is a file called ui.properties. Edit that file. The following line is in that file: “#serviecPort=4243”. This line is commented (#), because 4243 is the default value. You can leave that line commented and add a line below it:

servicePort=4200

(You could also remove the comment and replace 4243 with 4200, but I recommend adding a new line.)

Save the file to disk. While still on laptop, open a terminal window and execute the following SSH command (if you’re using Putty to do this on Windows, rather than cygwin, I recommend reading the post back on the CrashPlan site).

ssh -L 4200:localhost:4243 user@192.168.5.71

In the above command, substitute your username (or root) where I put in “user” and the IP address of your PogoPlug where I put in the “192.168.5.71”. This command makes port 4200 on laptop magically redirect to port 4243 on the PogoPlug, which is where the CrashPlan server is listening by default already.

Now launch the CrashPlan GUI on laptop by double-clicking the icon (on Windows, it’s probably in your system tray). You should see a request to create a new CrashPlan account (free) or log in to an existing one. Since I already have one (and presumably you do too), just put in your email address and password. It took quite a while for it to log in and download the configuration, but it worked. I think when I first tried it on my first PogoPlug it actually timed out, but worked the second time.

Once that’s done, you can exit the GUI, as all of the defaults are exactly what we want/need. The only exception to that is if you want to let others (that aren’t part of your account) backup to your PogoPlug. Then you will need to write down the CODE to enable that (it’s toward the bottom of the front page on the GUI and also on a Settings tab).

You can now exit from the ssh session that was started above (Type: exit or hit Ctrl-d in that terminal window).

Once the GUI is shut down, edit the ui.properties file again and delete the extra line we added with “servicePort=4200” (or place the comment “#” back in front of it). Save the file.

Launch the GUI again. This time it will connect to the local CrashPlan server on laptop. Now click on Destinations (bottom left entry on the left-hand navigation). Then click on the icon labeled Computers in the center (the PogoPlug is a real computer!). Whatever you called your device should be in your list, no code necessary, since you should have used the same account on both machines. If you didn’t name your device, then Arch Linux ARM defaults your host name to alarm (get it? ArchLinuxARM?).

Now you’re truly done. If you have a large amount to backup, it could take a couple of days to complete the first backup, even though it’s on a LAN. It will also alternate between the various backup locations (including CrashPlan Central), which is one of the reasons it will be somewhat slowed down.

For reasons I can’t explain, it can take a very long time to start the initial backup, even if you pause the other locations. The GUI correctly communicates with the server instantly, since I can see the correct directory created on the PogoPlug, but the destination still shows up as unavailable for some period of time. Eventually, it gets going, and appears to be quite reliable from that point on.

Update: I struck out the previous paragraph since I’m hopeful that with the addition of libjnidispatch.so to the jna jar file, you won’t experience the long startup delay.

cp -p jna-3.2.5.jar jna-3.2.5.jar-ORIGINAL

Trust the Sidux.com Home Page

Send to Kindle

If I detailed all of the hurt I caused myself (entirely my fault) in this one post, I’d be typing for hours. I’ll spare myself (and you) all of the gory details, and summarize.

If you follow my techie posts (way fewer than my music posts), then you know that I’m a big fan of Sidux. Most recently, I had settled on the Xfce flavor (for speed and simplicity), and I run it all under VirtualBox on my Windows Vista Ultimate x64 laptop.

I built the original installation on a dynamically-sized virtual disk, maximum size 4GB. That seemed like plenty of space, since Linux is not my primary OS, just a place to play around in, and to be safer when appropriate.

For yucks, and mostly because there’s nothing production about my installation, I got into the habit of keeping up-to-date on the bleeding edge. That meant regularly typing:

apt-get update

apt-get upgrade

apt-get dist-upgrade

I think the middle step is unnecessary, but I do it more often than not anyway. Up until recently, I never had a problem with that.

Back to the disk space situation. I had over 1GB free, for a long time, which seemed like plenty of headroom, given that I don’t really create new content on Linux. Then I decided to experiment with a few apps on Linux (some of which are only available on Linux), for HTML editing and other web creation tools.

Once I installed a bunch of them, and continued my dance of dist-upgrading regularly, my disk space got closer to the 4GB max, without me paying any attention to it. Then, during one dist-upgrade, I temporarily ran out of space (during the actual upgrade, while extra space is necessary to unpack things, run scripts, etc.). One or two steps in the upgrade failed to install (due to disk full errors), but most of the upgrade went fine. In fact, I had 86MB free after the ugprade, and things appeared to be OK.

That’s all background to my comedy of compounding errors that ensued. It began with a bit of hubris (too much cleverness on my part). Knowing that I just dist-upgraded myself into a full disk, and knowing that to correct it, at a minimum, I’d need to run a normal upgrade, I decided to shutdown Sidux, rather than reboot it, thinking I would avoid any booting problems with the just-failed dist-upgrade.

I then searched for how-tos on growing a VirtualBox disk (having some instincts as to what I could do even if I didn’t find an answer). I found some multi-page forum posts that confirmed my instinct.

I created a new fixed-size 8GB virtual drive. I then attached the old one and the new one to an instance of CloneZilla (under VirtualBox, obviously). I let it do a disk-to-disk copy. Unfortunately, I also told it to grow the partition on the target drive to fill all of the space. I say unfortunately, because the target disk ended up being full at 8GB, so something went wrong on that end.

This is one of the points where I’ll spare you tons of gory details. Suffice it to say that I eventually got the target disk to be a perfect clone of the original disk, with the additional 4GB of free space now correctly recognized.

I finally was ready to attach the new drive to the Sidux VM, and boot it up, expecting to do an upgrade afterwards, and be back to normal. Wrong…

When I booted, I saw a different Xfce splash screen. The dist-upgrade had given me the new Xfce 4.6, and (unbeknownst to me) also the Xorg 1.6 upgrade. I ended up with a working X installation, but a non-working Xfce one. In other words, the graphical environment was up, but I had no menu system (at all), with no ability to right-click on the background to bring one up. The only working icons in the dock were the web browser, and the log off one.

Here’s where I ended up wasting a stupid amount of time. Basically, I assumed that my clone of the hard drive failed, so I went down a few wrong paths trying to correct that. I even went so far as to install from the Sidux CD over again, onto the new (reformatted) 8GB drive. But, like a complete ass, I immediately dist-upgraded after the install.

Guess what? My system was hosed exactly the way it was before! That made me boot off my old, full 4GB drive, and sure enough, same menu problem. So, the clone of the disk went fine, it was the dist-upgrade that had screwed me.

But, I was still too thick to realize that. I thought (at this point completely foolishly!) that it was the fact that the disk filled up during the dist-upgrade that caused my problem (a now ridiculous assumption, since the same problem occured on a fresh install into a large disk). So, I wasted a little more time trying to remove and reinstall certain packages, all to no avail.

Then, I finally followed the advice I give in the title of this post: Trust the Sidux.com Home Page!

Right there on the home page, they tell you exactly when it is not safe to do a dist-upgrade, and why. Of course, just my luck that I happened to do a dist-upgrade at the wrong time, and, at the same time, happened to run out of disk space, causing me to misdiagnose my problem!

Is there a happy ending to this story? Yes, but I’ll skip a lot more pain and cut to the chase.

I never did get Xfce 4.6 running correctly, even long after the Sidux.com home page said it was OK to do a dist-upgrade again. Instead, I decided to take advantage of another announcement on the home page, claiming that KDE4.2 was now available (for a while, it too was the cause of a no dist-upgrade warning). I have always liked KDE and only avoided it to avoid bloat. Given that I couldn’t easily get Xfce working again, I decided to give the new KDE a try.

While I have some complaints about the menu layout, I basically liked the look of KDE4, and I liked the quick launcher, which made the menu  layout less important to me. That worked fine for a couple of days, and then I did another dist-upgrade (no warnings against it on the home page!), and I lost X completely. I could only log in on the command line.

After a little poking around, somehow, during the upgrade, all of my display managers had disappeared from /usr/bin. In this case, specifically, there was no /usr/bin/kdm. No idea how that happened, but doing an apt-get install kdm solved the problem, and I’m now back in business. I can even do a dist-upgrade and everything continues to work, so I appear to be beyond the previous problem.

It’s quite possible that all I would have needed to do is to install xdm (like I had to install kdm) to get Xfce running again. I might try that in the next few days if I get some time.

Anyway, even though I had a ton of frustration over the past 10 days, none of it was the fault of Sidux. In fact, they tried to save me from myself, something that others have failed to do many times in the past as well.

Another learning experience is in the tank now, and another happy ending, since I don’t mind having experienced the new KDE either. 🙂

OpenVPN in VirtualBox

Send to Kindle

There are a number of reasons why you might want to run a VPN on your laptop. The most obvious is that you have to, in order to access files back at the office. Two other likely reasons are security (you are in a public place, and want to encrypt all traffic) and unfettered access (your provider is blocking certain ports or services).

I’ve been interested in implementing a VPN for a while now, but haven’t had any real need, nor tons of copious free time. On Friday, I was informed that one of my portfolio companies had installed OpenVPN and I was welcome to install a client and test it.

I am running Windows Vista Ultimate x64 (the x64 sometimes being problematic with certain software). It turns out that the latest release candidate for OpenVPN has full Windows x64 support, so that wasn’t going to be an issue.

I installed the software, edited the configuration files that the office sent me, fired it up, and it worked the first time. Cool. I tested a few things, including accessing hosts that I wouldn’t be able to see unless I was in the office, and it all seemed to work correctly.

Then I hit a small snag. I tried to fire up my fat-client brokerage application. It behaved as if I didn’t have a network connection, or more accurately, like it couldn’t find the server it wanted to home to. I suspect that this could be something as simple as whether the office itself is set up to route out this type of app/port/protocol through the VPN (I know for a fact that this specific app works when I am in the office).

I also suspect that other fat clients, like a Poker app, might have similar troubles. That got me thinking about the additional use cases beyond just needing access to files/apps/machines in the office.

I fired up VirtualBox, specifically with my new favorite Sidux distribution. I tried to install openvpn from the repo, but it wasn’t found. A Google search said that openvpn is included in Debian itself (which Sidux is based on), so I was temporarily puzzled. I had the following line in my sources:

deb http://ftp.us.debian.org/debian/ sid main

I added another line, identical to the one above, substituting de for us. Presto, openvpn was found, and installed smoothly.

I copied over the same config files from my Windows directory, fired up OpenVPN, and was connected to the office again. This time though, in a pretty cool configuration. Everything in Linux (Sidux), was routed through the VPN. Everything in Windows, was routed normally though my FiOS connection.

If I wanted access to something on the corporate lan, use the browser in Sidux. The brokerage app just worked as normal, as it was unaware of the VPN. On a number of levels, this is the best of both worlds.

Of course, I already summarized situations when you may want/need the full VPN, for the entire machine, or when this use case might be better. If you’re in a public hotspot, and want everything encrypted, even your personal surfing, the Windows-level VPN makes sense.

If you’re in a client’s office, and can’t connect to an odd port on your home server (e.g., you have an application running at http://www.mycompany.com:8765/) which is blocked by your client’s firewall, then you fire up the VPN in the VM, and use that browser, while not disturbing the rest of the applications on your desktop.

This also gave me the idea that since putting Linux on a USB stick is so trivial (see this post about multi-boot USB), it would be simple to have a bootable USB stick, with the OpenVPN client on it (password-protected, of course), that would allow you to boot off any PC/laptop as if you’re in the office, or without leaving any trace on the host PC, whenever the situation called for it. Friends wouldn’t need to feel that you were seeing their browsing history, etc.

Just for yucks, I also installed OpenVPN on my server, for the secondary scenarios mentioned above (security and unfettered access). While I don’t anticipate needing them frequently, knowing that it’s available, on a second’s notice, is a comfort.

Another trick added to my bag. 🙂

Sidux Wins Again

Send to Kindle

Almost two years ago, I wrote a post about an ancient (and very broken) laptop. Of the various Linux distributions that I tried on it, I really liked Sidux the most. I wrote about it in my In Praise of Sidux post.

I ended up trashing that laptop when the unreliability was more annoying that the brief moments that it would actually work (entirely a hardware issue, not a Sidux problem!).

A while later, I loaded a number of distros under VMware Player on my old XP laptop. Of course, Sidux was one of them. Unfortunately, I had problems getting X to work at greater than 800×600 (don’t know if it was a VMware problem, or a Sidux one, but as I noted in this post, I didn’t need it badly enough to track it down).

I’ve recently written about Virtualbox and how I got it to work with a multi-boot USB drive. In that post I mentioned the two main reasons that I boot Linux in a VM. I left out a use that is perhaps better, though I haven’t been disciplined enough to actually do it frequently. It’s almost the ideal way to surf to potentially dangerous websites, in particular, if you’re using a Live CD iso image to boot from. There’s simply nothing to infect on the part of the bad site!

Given how many malicious sites there are out there, it’s something I considered doing more often. In preparing, I decided that I wanted a tiny distribution, since I didn’t need to do actual work in Linux (e.g., I didn’t need an office suite, etc.). That said, I wanted two things:

  • Latest Firefox
  • Ability to build the VirtualBox Linux Additions

Both of those conspire against using something like Damn Small Linux (DSL, which I like), because it tends to use Firefox 2.x. I read a bunch, and Absolute Linux (12.2.1) sounded pretty good. I got it running quickly, and was even successful in getting the VirtualBox additions installed. I ended up giving up on it reasonably quickly for two reasons:

  • I couldn’t get the resolutions to be as flexible as I wanted, even with the additions installed
  • Package management was quite sparse and I wasn’t interested in going down the path of building tons of packages from source

In the past, I had success with Puppy Linux. I downloaded 4.1.2 and liked it instantly, much more than the 2.x and 3.x series that I had used before. Very attractive, very fast (booting and running). I really liked the unionfs filesystem. After trying reasonably hard to make this one work, I gave up (also for two reasons):

  • I couldn’t get Xorg to work under VirtualBox, but Xvesa worked flawessly
  • When I booted Puppy natively (from a USB drive), it couldn’t handle my Intel 5300 (a/b/g/n) wireless card (though NAT worked under VirtualBox perfectly)

Xorg worked flawlessly in native boot. Not having it work under VirtualBox meant no seemless mousing between Linux and Vista, a non-starter for me. VirtualBox couldn’t even find Xorg. 🙁

I hesitated to even look for Sidux, because I didn’t want a DVD-sized ISO file. Reluctantly, I went to the site anyway, and found that the 2008.4 release had multiple versions, including a 395MB CD ISO with Xfce instead of Gnome or KDE. That was very attractive to me, as I’ve liked the simple and clean interface of Xfce on other smaller distros, and I didn’t have a need for a more complex framework for multi-app work.

I downloaded the ISO and booted it in VirtualBox. Everything worked perfectly, instantly. When I say everything, I mean everything. I wrote a post a while ago about how Ubuntu worked out-of-the-box under VMware Player, and I didn’t understand how. Now I do. The VirtualBox additions are already built in with Sidux (or, perhaps, VirtualBox recognizes Debian, and supplies the correct drivers to fool the operating system).

The point is that I could definitely run Sidux as a Live CD if I wanted. Pretty darn cool. But, I decided to install it to a virtual disk anyway. This way, I could have a customized installation with my SSH keys, aliases, plugins, etc. It would also make it less painful to upgrade to the latest versions of packages (instead of waiting for the entire distro to be updated on a new CD).

So, I installed it, and the VirtualBox additions (because I wasn’t sure whether the latest version, 2.1.0 was there by default). It’s simply fantastic. I can copy/paste across Vista and Linux. I can move the mouse seemlessly between the desktops. I can change the resolution if desired, including going to a full 1920×1200, going full screen, making the machine appear to be a native Sidux Linux one (Vista simply disappears completely). Then, without rebooting, I can just change the resolution back to 1400×1050, which fits nicely within the Vista desktop.

I have shared folder support (which I mount at will, so a virus can’t infect Vista since I only mount if I need to move a file from one environment to the other). I have full USB support to the virtal machine (so I can read/write from a USB stick from Linux). Like I said above, it all just works.

So, while I am glad that I learned a bit about some other distros (in particular, Puppy 4.1.2 which is really great), Sidux wins again for me. It’s simply a fantastic distribution.

Recovering From a Server Disk Crash

Send to Kindle

The machine that is delivering this blog to you is a standalone Dell server, running CentOS 5.2. It resides in a data center managed by Zope Corporation (ZC) Systems Administrators (SAs). I perform the majority of the software administration and maintenance myself, and they maintain all of the hardware along with some of the software.

The single most important software maintenance that they are responsible for is backing up the system. We’ll get to that in a minute.

This past Sunday, we were in our usual hotel room. I was logged on for roughly eight straight hours, and all was well on this server. I shut my laptop off at around 10:15pm. When we woke up in the morning, Lois checked her email on her Treo. It kept hanging. We thought it was a Treo/Sprint problem, but even rebooting the Treo didn’t correct the hang.

When we got into the office (at around 7am), our laptops couldn’t retrieve mail from the server either. Pinging the server worked, but I couldn’t SSH on to it. In fact, most services delivered by this server were gone (including this blog, which was unreachable). The only service (aside from ping/ICMP) that was obviously up was Zope! If you just wanted to visit our corporate site, and learn about our VC business (all of which is handled by Zope), that was running fine!

The head of the SA group at ZC was also in that early (thankfully!). After poking around a bit, I encouraged him to power off the machine remotely, and them power it back up remotely. He did. Unfortunately, the machine didn’t come back up (or at least it wasn’t responding to pings any longer, so something was wrong).

Another SA was called and directed to go to the data center rather than the office. When he got there, and hooked up a console to the server, he saw that the disk was failing. Attempts to get it started (running fsck, etc.) proved fruitless. It decided to die completely that morning.

Yet another SA was dispatched (from the office) to the data center with a fresh disk (he had other work to perform at the data center, or I would have delivered the disk myself, happily!). We knew in advance that this disk might be slightly problematic, as it had CentOS 4.6 installed on it.

When the disk was inserted into the machine, it booted up immediately. I was able to successfully SSH to the machine, but of course, nothing of mine was on there. That’s when the original SA (who also happens to be our expert in backups) started restoring the machine from our backups.

For a long time, we used to run the Amanda backup program at ZC. I don’t know why, but our experience with Amanda was never good. I am not suggesting that others don’t find it to be a perfectly acceptable solution, but for us, for whatever reasons, it wasn’t good enough.

After searching and evaluating a number of alternatives, the ZC SAs selected Bacula. We’ve been using that for a reasonable period of time now. The Bacula restore of my machine didn’t take all that long, and every file that I wanted/needed was successfully and correctly restored. In fact, the nightly incremental backup had run successfully before the disk decided to die (how polite), so even some non-critical files that I touched on Sunday were successfully restored! Whew!

That said, it was hardly a trip to the candy store. All of the applications that I compiled myself (more than you’d think, I’m a geek at heart!), didn’t work because of the OS (operating system) mismatch. My programs required newer versions of various libraries (possibly even the kernel itself in some cases), so starting from a 4.6 machine and restoring files that required 5.2 wasn’t as clever as we’d hoped.

Still, with some pain, theoretically, one can ugrade a 4.6 machine to 5.2 over the network. That was my plan… Well, the best laid plans, as they say…

I released the SA from the task of baby-sitting my machine, because I knew that a network upgrade would take a while, at best. After doing some network magic, the machine was in a little bit of a funky state, but there was a chance that a reboot would bring the machine up in a 5.x state, making the rest of the upgrade fairly straightforward.

Unfortunately, when I rebooted, the machine hung (wouldn’t actually go away, still pingable, but otherwise non-responsive). Again I asked the head of the SA group to remotely power down/up. Again, it powered down properly, but didn’t come back up.

In fact, it likely did come up, but because of the funky state that I left the machine in, it couldn’t be seen publicly due to network configuration issues. This time, we decided to take a more conservative approach, because opticality.com was down for at least 8 hours already (not a happy situation for either Lois or me).

The original SA went back down to the data center. This time, he burned a CD with a network install ISO of CentOS 5.2. After installing the correct OS onto the machine, he again restored the disk with Bacula. This time, everything matched. Still, there were problems…

The biggest issue (by far!) was foolishness on my part in mapping out what I wanted backed up on the machine to begin with. Sparing you the gory details, I ended up restoring the Yum database from my backup over the actual Yum database from the installation, so the system didn’t really know what was installed and what wasn’t. Not a good thing.

I only really cared about email to begin with. I built a few things (pretty quickly) by hand, and got email running. Then I got the web stuff up pretty quickly too. Finally, IM. Those are my big three hot buttons, everything else could be dealt with later on.

I didn’t want the SA to leave until we could prove that the machine could be booted correctly remotely. That took some time as well, as a number of the services that are started automatically weren’t installed on the machine (though RPM/YUM thought they were!). We (or rather he, at the console) disabled them one by one, until the machine came up.

After I restored the more critical ones, we tested a reboot again, and it came up fine. Whew. I released him again, this time for the last time.

I cleaned up a few more things and went to bed reasonably happy (it was now close to 10pm on Monday night). Over the next two days, I spent a few hours cleaning up more things. Yesterday, I completed the cleanup.

A series of shell scripts and filters, doing things like the following:

yum list | grep installed | cut -d ‘ ‘ -f 1 > /tmp/installed

Then running the resulting packages (the ones the system thought were installed!) through:

rpm -V package-name

Filtering that output for lines starting with the word “missing”. Then removing those packages (they weren’t there anyway, so it was a database cleanup activity) and then installing them via Yum again. It wasn’t actually as painful as it sounds, but it wasn’t pain free either.

The biggest headache occurred when removing a non-existent package also moved a config file (that was being used by a package I built separately, so Yum was unaware of it). At one point yesterday, without realizing it, I killed our SMTP (email) server. Oops. We were down for about 10 minutes, before I realized it, and got it back up and running.

At this point, all is back to normal, and Yum correctly knows what’s on the system.

Here are the lessons:

  1. Back Up Regularly
  2. Have a great group of dedicated SAs behind you
  3. Have some spare parts close by
  4. Think hard before taking some stupid actions (all my fault!)

I’m truly amazed, pleased, impressed, etc. that we lost nothing. Of course, we were down for 12 hours, but Internet email is a truly resilient thing. All mail that wanted to be sent to us was deferred when our SMTP server didn’t answer the call. When we came back up, mail started to be delivered nearly instantaneously, and by morning, all mail had been correctly delivered.

Here’s hoping I don’t go through this experience again, and here’s hoping that if you do, this post might help a bit… 🙂

Rube Goldberg and SSH

Send to Kindle

Rube Goldberg would be very proud of what you can accomplish with SSH. If you don’t know what SSH is, you really should just stop reading, as not only will this post be meaningless to you, you wouldn’t care about the result (or technique) even if you followed it perfectly! 😉

A while ago, I gave an ancient laptop to a good friend who was sharing her husband’s laptop while they lived in Princeton for a year (he just finished a fellowship there and they are returning home in a week). Given the age of the laptop, the amount of RAM, the size of the hard drive, etc., I was hoping my friend would be willing to live with Linux rather than Windows on the box.

She’s a Gmail user, so she was indifferent (obviously, having never tried Linux before), given that she basically lives in the browser most of the time she’s on the machine. I installed PCLinuxOS on it (the 2007 release) because it looks a bit more like Windows than some other popular Linux distros. Today, I would make a different choice, not that there’s anything wrong with PCLinuxOS.

Anyway, for the most part, it has worked out very well for her, and she’s felt connected to the world, especially when her husband was in the office, and there was no laptop at home. Unfortunately, there is a bug somewhere either in PCLinuxOS, in her hardware, in their Linksys router, or in the timing between all of them, because on occasion, when the laptop boots, it doesn’t get the correct DNS server info even when it gets a good IP address via DHCP.

In what has to be a very painful process for my non-techie friend, I have given her a series of a few commands to type into a terminal window to correct the problem, when it occurs. Of course, it’s entirely Greek to her (rightfully so), but it works, and she patiently types them away and it all magically starts working again.

On occasion, she’s had trouble getting all of the commands in correctly, and she feels guilty calling me, even though I keep telling her that I don’t mind helping whatsoever. That got me to thinking about how I could fix it permanently, without making her life even more miserable trying to talk her through the various things I’d like to try in order to be sure I found the right solution.

I don’t have time to visit Princeton this week, and soon, she’ll be back in Indiana, and I definitely won’t be visiting there in a while. So, I need to have remote access to her machine. I can’t just SSH into it, because I certainly don’t want to talk her through port forwarding on her Linksys router, nor do I want to leave that port permanently open. That cuts out a vanilla VNC connection as well (which would be overkill, but if it was available, would work as well).

So, I thought that perhaps I would try one of the web-based remote control services. I have had excellent success with the free service from Yuuguu.com when I help my Dad with his Windows machine. It works on PC’s and Mac’s, but apparently, not yet on Linux, even though it’s Java based. That was disappointing. A peek on a few others yielded similar results.

After scratching my head a bit, and searching the net a bit more, I came across a very simple solution, entirely via SSH, but with Rube Goldberg implications in that I was solving a very simple problem, with a built-in option of SSH, but jumping through tons of hoops to get to the point where the simple command could be issued.

The solution (tested by me, but not yet done with my friend, because I wanted to be sure before subjecting her!) is as follows:

I’m running Windows XP. I could run an SSH daemon there (in a number of ways), but since this is a temporary solution, which I don’t really want to think about, instead I fire up VMware Player and launch my new favorite mini-distro, CDLinux 0.6.1. It automatically fires up an SSH server.

I then poke a hole in my firewall (I didn’t need to talk myself through it either) 😉 with an arbitrary port number (for argument’s sake, let’s say it’s 12345). I forward that to port 22 on my CDLinux instance (running under VMware, and therefore having a different NAT’ed IP from my Windows box!). I can even leave the firewall in that state permanently if I want, since 99.9% of the time, CDLinux won’t even be running, and even if it was, and someone luckily got in, it’s a Live CD image, with nothing to really harm!

OK, we’re almost done! On the remote machine (my friend’s), she would type the following, in a terminal window:

ssh -l cdl -p 12345 -R 54321:localhost:22 the.name.of.my.remote.machine

She’ll get redirected to my VMware instance, and be prompted for a password, which I’ll give her in advance (can’t use ssh keys for this, since I don’t want to over-complicate this). Once she is in, I open a terminal window in my instance, and type:

ssh -l her_user_name -p 54321 localhost

Voila! I’ll now have a shell on her laptop, through an SSH tunnel, without her poking any holes in her firewall, and without me even needing to know her IP address, unless I want to restrict my SSH port forward to her specific machine, which would make this dance even more secure.

I’ve tried this a few times from different machines, all with success, but my friend isn’t online at the moment, so the final test will likely have to wait until tomorrow morning. In any event, a cool (and relatively simple solution) to an otherwise thorny problem. Just as a footnote, if I needed more control over her machine, the exact technique could be used to reverse tunnel a VNC port, giving me graphical control, or I could SSH back with -X (for X-Windows tunneling), and launch graphical clients one at a time, etc.

Update: OK, so today we got to try, and it worked perfectly. The only kink was that sshd was not automatically started on her laptop, so I had to talk her through becoming root and starting the service (simple enough!).

After we got it going, I did the unthinkable, and offered to upgrade her system. It was reasonably old, with Firefox at 2.0.0.7 (for example), and lots of other packages that could probably stand a security update. I warned her that it’s often better to leave these things alone when they are running smoothly, but in the end, we both decided to go for it.

So, I ran an apt-get upgrade. I then asked her to reboot. The machine came up, but only in terminal mode. She was able to log on, but startx failed with tons of errors. Oh oh…

Thankfully, the network did come up, and she was able to log on and run the ssh tunnel. I was then able to get back on her machine. I decided that instead of poking around too much, I’d try one last thing, which was to perform an apt-get dist-upgrade. This ran for a bit, and then I asked her to reboot again.

Voila! The machine came up correctly, and the networking worked again. So, for the moment, it seems that we accomplished everything we set out to do today, including her running Firefox 2.0.0.14 (I know, not 3.0 as yet…). Whew! 🙂

Updated Linux Distros in VMware Player

Send to Kindle

I’ve written before about running Linux under Windows XP using the free VMware Player. It works really well. Even though I’ve done it before, I don’t really have much of a need, so other than making sure most big-picture features work, I don’t really exercise the distribution.

Recently, I’ve had two reasons to crank it up just a drop (literally, just a drop, I’m not yet using VMware Player for anything serious). First, the possibility (however distant or unlikely) that my next laptop may be running Linux as the primary OS. Second, there have been a flurry of new (updated) Linux distros released this month, some that I have had a long curiosity about.

In the past, I’ve had little more than glimpses of Ubuntu releases (6.06 and 7.04). I didn’t really give either a whirl, but my initial impression was less than enthusiastic. The color scheme alone (I know, easy to change) was muddy and boring looking. On a more important note, I have always struggled (with little information!) as to whether there is a material difference between choosing a Gnome-based distro, or a KDE one.

To my eye, KDE looks better, but as much as I enjoy eye candy, it’s not the over-riding reason for me to select an OS (or I’d be happy with Vista, or I would have run to a Mac). If Gnome is more functional, or has a more likely future, I’d happily put up with a less-pretty UI, and even put up with less user friendliness.

Recently, I read a review of a late beta of Ubuntu 8.04 (Hardy Heron). The guy raved about it. In the past, I noticed that it took a day or two for KUbuntu (and other derivatives) to be released after the main Ubuntu distro, and that made me feel that they were step-children, possibly not as robust or integrated.

This time around, all of them were released on the same day, including a KDE4 version of KUbuntu as well.

So, in April alone, I downloaded and tested the following Linux distros:

  1. Sidux 2008.01
  2. Ubuntu 8.04
  3. KUbuntu 8.04
  4. KUbuntu-kde4 8.04
  5. DSL 4.3 (Damn Small Linux)
  6. SystemRescueCD 1.02
  7. CDLinux 0.6.1

Sidux (last year’s flavor) was one of my favorite distros. It’s based on Debian (as is Ubuntu) but it is tied to the unstable repository so you get more frequent updates (of things like Firefox for example). The 2008.01 release is a DVD iso, all of the other ones mentioned above are CD isos.

While it booted up fine (in Live mode, under VMware Player), it was not able to run in any resolution other than 800×600 (the default). That’s not entirely true, it could be made smaller, not larger). I hand tweaked the xorg.conf file and tried a few other things, none of which worked, and I quickly gave up. Remember, I don’t really have a short-term need, so struggling wasn’t appetizing and I had other distros to check out anyway.

I have installed every version of DSL for quite a while, so adding 4.3 to the mix wasn’t a surprise. It’s a good distro for getting small jobs done. One thing to keep in mind (not necessarily a downside) is that it’s still based on the 2.4 kernel branch. Anyway, this one works just fine. If it wasn’t for the next distro I am about to cover, this one would get some use from me whenever I needed an X Server on my desktop.

CDLinux 0.6.1 is the latest version of CDLinux (Compact Distro Linux). I hadn’t heard of it before this release. It’s a little larger than DSL (about 10MB bigger), but it still clocks in at under 60MB. What intrigued me was that it is significantly more modern. It uses the latest 2.6 kernel, Xorg, XFce (window manager), the latest Firefox (2.0.0.14), etc. I have to say that I really like this one for quickie jobs. It’s clean looking.

I am writing this post on CDL (under VMware Player) running Firefox. I scp’ed over a certificate for Firefox and I am using OpenID to log in as me to WordPress. I’m running the Live CD image, so my disk drive is ram. It’s working delightfully well. My only semi-complaint is that at the resolution that I’m running it (1400×1050) the fonts aren’t all that attractive. I don’t know if that’s a CDL issue, an XFce one, a resolution only one, etc. I don’t really care at the moment, but I thought I’d mention it.

A quick mention of SystemRescueCD. That’s another one (like DSL) where I download each version, and have been doing so for quite a while. It’s a very nice emergency CD, and while I rarely need one, this is the first one I turn to on those rare occasions. The only thing I do when I download a new version is check that it functions correctly under VMware Player, then I burn a real CD, as this one is for real emergencies, not for playing around in a virtual machine.

Now the Ubuntu family. My first impression (also not detailed in any way) is also extremely positive. Even the less-attractive main Ubuntu (Gnome-based) is reasonably nice. The KDE one and KDE4 one are both more attractive. While the KDE4 one looks very nice, I’m not sure that I don’t prefer the look of KDE3. I have no problem with KDE4 and could easily get used to it, and perhaps the only reason I prefer KDE3 is that I’m already used to it.

As opposed to Sidux, the rest of the distros mentioned above all resize easily and flawlessly to any resolution I like. Cranking them up to 1400×1050 was trivial, and worked immediately. My native resolution is 1600×1200, so I have plenty of room to run a 1400×1050 sub-window for Linux.

One curiosity. All of the Ubuntu distros automatically release the mouse at the borders of the VMware window. This is a default behavior that I prefer, making the Linux window feel like just another app on my XP desktop. The only theoretical downside is that alt-tab doesn’t cycle between the windows within Linux. The other distros (including CDL which I’m currently using to write this) trap the mouse at the borders, and force me to press Ctrl-Alt to release the mouse. It’s not that big of a deal, but I am curious as to what each distro is doing, as none of them knows about VMware.

Anyway, all are fine distros and may see more time on my desktop over the coming months. Like I said above, while it’s still not likely, there’s a possibility that my next laptop will be Linux, and my primary distro will either be one of the Ubuntu flavors or Sidux.

Microsoft Madness

Send to Kindle

Yesterday, I read the following article on PC World’s website. It mirrored my thoughts about Windows XP vs Windows Vista perfectly, including direct experience not just theory.

What I learned in that post (which I probably should have known earlier but didn’t) is that Microsoft intends to stop most sales of Windows XP as of June 30th, 2008. I’m not really sure what most means in this context, but either way, it’s boneheaded.

I just did a quick search, and apparently it means that they likely won’t be offering it to OEMs, so if you expect to get Windows pre-loaded on a new laptop after June 30th, you’ll have a choice of Vista or Vista (or Vista or Vista, given that there are four version of Vista available!).

John Heckman questions whether Microsoft won’t bow to pressure and push back the June 30th date.

The minute I read the article I knew I was going to post this. My first instinct was to title it Wake Up Microsoft. Then this morning, it came to me, this is the perfect season to aptly and correctly use the term Madness.

It’s clear that Vista is a bomb. You’d be hard pressed to find anyone without an ax to grind that would seriously defend the merits of Vista over XP. It’s not the first time Microsoft has bombed with an entire operating system. How many of you are still running Windows ME?

At least with Windows ME, it died a relatively quick and painless death. With Vista, for any number of reasons, Microsoft isn’t willing to give up. Given enough time (and money), they will likely make it decent, though it’s unlikely to ever be great (given it’s core), and it’s not even likely to get decent given that they are already working on it’s successor.

The madness isn’t in not killing Vista (I understand that the investment and marketing bets that they’ve made are too big to simply throw away). The madness is taking away the only viable choice that still puts money in Microsoft’s pocket!

Folks, there’s no doubt that XP is eating into Vista sales. That’s the only reason that Microsoft wants to stop selling XP, they want to remove the competitive choice and force new computers to be pre-loaded with Vista! Will it work? Of course, there are many people who wouldn’t consider Linux or Mac under any circumstance, and they will grudgingly (or ignorantly) accept a machine with Vista on it, if they have no other choice.

This doesn’t make it a smart strategy. The sane move would be to keep offering XP as a choice (while heavily promoting Vista). Then, whenever Vista truly rivals XP (don’t hold your breath), or Windows 7 (or whatever it will be called when it finally arrives) is available, stop selling XP.

In the best case scenario, Microsoft will sell exactly the same number of licenses in total (Vista only, instead of a mix of Vista and XP). They will get to declare a huge PR win for Vista (look how sales ramped so nicely!). They will not get any additional profit (since they will be maintaining XP for years to come anyway). They will create a slew of miserable users who will equate Microsoft with pain (or worse).

In the worst case scenario, they will push people toward alternative operating systems like Mac and Linux.

I haven’t done a scientific survey, but I honestly believe that nearly every technology professional (business people too, not just developers) that I know has switched to using a Mac as their primary computing platform (most on laptops, but I know a number of people who use iMacs as well!). When I say “nearly every” one, I believe the number is pretty close to 90%.

Examples include Zope Corporation. While 100% of our services to customers are delivered on Linux-based servers, there is only one developer in the company that hasn’t switched to a Mac. Even the SAs (System Administrators) all got Macs recently (though one of them decided after the fact that he’s more productive on his Linux laptop).

My friends (you know who you are) have been needling me for years to switch to the Mac. I have very long experience with the origins of Mac OS X (NeXT), so no one needs to convince me of the power and the beauty of the underlying software.

I haven’t switched for two reasons:

  1. There are programs (some cool, some necessary) that only run on Windows, or at the very least, run on Windows way earlier than they become available on Mac.
  2. The value proposition of generic hardware (laptops and desktops) is overwhelming vs the Mac stuff. The Mac stuff is gorgeous, and brilliantly designed. Ultimately, it’s not worth the money and locks you in. They also have enough quality problems to make me pause.

My non-technology professional friends (neighbors for example) still prefer Windows. There are a number of reasons but they are all valid (games for their kids, Windows is used at the office, I know Windows, I don’t want to have to buy new copies of software I already paid for, etc.).

In April 2004 I bought my current laptop. In fact, I just wrote about that in this post. I bought it without an operating system pre-loaded because I was committed to switching to Linux full time. The experiment lasted six weeks (not too bad), but once I started running Windows in Win4Lin, I realized that I wasn’t quite ready to cut the Windows cord full time, and I installed Windows XP Pro.

There were two reasons that I switched back:

  1. 95% of the day I was happier on Linux than on Windows. 5% of the day I required a program that was only available on Windows. That 5% started to bug me more each day until I switched back.
  2. Linux was great in 2004, but it wasn’t quite as good on cutting edge hardware as it is today, and I had some real problems on my (at the time) brand new beast. It’s possible that I would have toughed it out if Linux had worked perfectly on my laptop back then. I have no doubt it would work flawlessly today.

My one direct experience with Vista came when my next door neighbor bought a new Dell Laptop for her mother. There was no choice, Vista only. I am their tech support team and she asked me to customize the machine for her mother when it showed up. I was amazed at the hoops I had to jump through to install programs onto the machine. I couldn’t begin to imagine what someone who was less technical would have done (other than throw the machine out!).

In addition, the machine crashed on me at least 10 times in one day during the setup. Sheesh.

Since then, I have been asked for laptop recommendations at least five times. In all cases, the buyer wanted Windows. In all cases I have vehemently recommended XP, and (amazingly enough) it was now available again as an option. None of those users has had a single problem with their new laptops.

Where does that leave me? As I mentioned in my spring cleaning post, I will likely be buying two new laptops at some point (possibly this year, but definitely next year if not in 2008). I have thought about this (before knowing about the demise of XP) for much longer than I care to admit, and I decided that I was going to stick with Windows. Sorry Mac fanboys. 😉

If Vista is my only choice, I can guarantee you that I won’t be buying it. Best case scenario (for Microsoft) is that I will buy a retail CD of XP and load it myself. Much more likely scenario is that I will install Linux on the machine, and try really hard to avoid the few Windows-only programs that I’ve come to rely on. The least likely choice is that I will break down and buy Mac laptops, but it’s not impossible (the possibility is at least on my radar for the first time ever).

So, coming full circle to my original post title: Wake Up Microsoft!

Who Needs Floppies

Send to Kindle

Since I just wrote about my laptop spring cleaning, I may as well get one more geek post out of my system. 😉

I run many Asterisk servers. I love it. That said, I am still running the 1.2.x branch on all of the servers. They are up to 1.4.18.1 on the production branch. I will never install the 1.4 branch. Not because I don’t believe it’s good, but because they are getting close to releasing 1.6 into production (they are currently at 1.6.0-beta6!).

So, I was interested in getting a test machine set up to install it (after it goes production), so that I can get to know it before committing it to production servers. I considered running it on a VM on my laptop, but I really want to avoid that if I can (read my spring cleaning post again for any number of reasons).

I considered buying a used machine on EBay, Geeks.com, Tiger Direct, etc. You can get pretty beefy machines for under $200, and reasonable ones for well under $100 on EBay, but you’re risking the seller, etc.

Last week, while at Zope Corp., I noticed that they were gathering old junk in an area for their own version of a spring cleaning. In that pile were two old machines. One of them was a Dell Dimension 4550, a 2.53Ghz machine, 30GB hard drive, with 256MB of ram. Not exactly the kind of ram you’d like to see, but otherwise more than adequate to power Asterisk. For a test machine, ideal!

I asked (multiple times) if anyone else hoped to snag it, or ever see it again. People laughed (rightfully so). 😉

Into the back of my SUV it went. I stored it for a week in our utility room and today I finally pulled it out. I wanted to install CentOS on it. The other day I downloaded the 3.6GB DVD ISO in a drop over an hour on my FiOS link. Yummy!

I popped the DVD in the drive and booted. Nothing, it just booted into the existing CentOS 4.2 (I wanted to install the 5.1 release). Hmmm. Thankfully I didn’t waste time figuring this one out. I quickly found out that the machine had a CD drive, no DVD. OK, moving on…

I downloaded and burned a CentOS net install CD (only 7.1MB) and booted again. Again, straight into the old CentOS. Hmmm. Somehow, the CD drive isn’t working (boot order was set correctly).

I didn’t have root access on the machine, and it can PXE boot (boot over a network, but I didn’t have a target machine for it to boot off), but it can’t boot off a USB device. 🙁

Floppies to the rescue! My second choice for an operating system was Debian. I downloaded five floppy images for a net install. I booted off of the floppy, and it failed again. This was getting very tiresome…

I booted into the existing system, and tried to mount and read the floppy. It took forever, but finally, I got a clean listing, so there was no hardware problem with the floppy. I tried that with a CD, but it was never able to mount that, so indeed, there is a hardware problem with the CD drive.

It turns out that even though I pressed F12 to change the boot order, and I picked the floppy, it failed. I pressed F2 (for yucks) to get into setup. Once I moved the floppy boot up the ladder, and saved, it successfully booted off of the floppy. Whew.

I now have a smooth running Debian system configured to my taste. I am now patiently awaiting the final release of Asterisk 1.6.0.

So, do we need floppies? Hopefully not going forward. But, as long as there is life in older systems (and clearly there still is), the fact that my four-year-old laptop has a built-in floppy drive ended up saving me some headaches. More important, are you impressed that I had five blank floppies handy as well? 😉