Tue Sep 30 14:45:01 CDT 2008

Paint by Numbers

I used to do Paint by Numbers when I was a kid. I'd end up with something that looked like a bad painting of a horse, but when you're working with 8 colors, what do you expect? Paint by Numbers works because it strictly limits your color choices. By removing the subtle difference between Mahogany and Oak and calling them both Brown you are able to work with a manageable palate. This also gives you the advantage of wide target areas to paint. "Chest is brown" is a lot easier than "Hair #7281 from the left is brown (shade #832) with black shadow (shade #19)"

Computers work in a similar way. You have probably noticed videos on Youtube that seem blocky and blurry. This happens when the computer breaks a picture into more manageable pieces to reduce file size.

Most image editing programs have a "Paint Fill" or "Bucket" tool that allows you to change the color of a section of the image. This normally doesn't work well with photographs because the color sections are too subtle. When you try to fill a section that looks white, the computer might see hundreds of shades of white in that area and only fill in exact matches. Dithered images have a similar problem. Dithering is dot shading, like in comic books. It is common in .gif images.

If you have a complex image and you want to simplify it to fewer colors, you can use The Gimp. The Gimp is an image editor like Photoshop, but it's Free software.

To reduce colors in an image:
  1. Open the image in The Gimp, using the File menu
  2. In the image's window select the Image menu
  3. From Image, select Mode, then Indexed
  4. Select "Generate optimum palette" and a Maximum number of colors.
  5. Experiment with the Maximum number of colors. Somewhere between 5 and 30 usually works.
  6. Make sure "Color dithering" is set to "None"
You might also note that after selecting "Indexed" you cannot set the options by selecting it again: First select "RGB" from the Mode menu then select "Indexed" again if you want to keep your new color scheme. Otherwise, you can select Edit > Undo (Ctrl + Z) to revert to the previous color scheme.

I used this to template color schemes for a web site. It is much easier to use the paint tool than to track down color codes in my css file when I want to try a dozen combinations in a hurry.

Posted by Nesman | Permanent Link

Thu Jul 10 20:00:53 CDT 2008

Google Docs

If you haven't seen it yet, you should check out Google Docs. Google has set up an Office Suite and made it available online, for free. I used it to create a spreadsheet to calculate how much money we spend on gas per year.

Nesman's Gas Prices Computed by Mileage

You can choose to share your documents with specific people, or publish it and make it available to everyone. You can create spreadsheets, documents, and presentations (read: Excel, Word, Powerpoint). These documents are connected to your Gmail account and accessible anywhere without having to install any software; let alone making sure to have the same version of the software.

I found the spreadsheet program easy to work with, and I'm not a big Excel user. (This may have been to my advantage) There is even an offline Google Docs mode so you can work without an internet connection. Why are people still spending hundred of dollars on MS Office?

Posted by Nesman | Permanent Link

Fri Jun 20 21:56:57 CDT 2008

MythTV - Technical

I'm running MythBuntu, based on Ubuntu 8.04 and using MythTV 0.21. I get standard definition, analog cable from NewWave (the local cable monopoly). My video tuner is a Hauppauge WinTV-PVR-150, which has a mpeg encoder built-in. (~$70) The encoder takes a lot of strain off of your cpu, freeing it up for other tasks like commercial detection. To get the picture to my TV, I have a EVGA GeForce FX 128MB PCI video card with DVI and S-Video out. (~$40) For my little TV, we'll be using the S-Video connection. Many of you could use the DVI connection for a newer tv, or even settle for an S-Video -> RCA adapter.

It helps to have a large amount of free drive space; default files take up about 2GB/hour. You can transcode the .mpg files to .nuv (NuppelVideo) which cuts file size in half and actually improves quality a bit. My .nuv files seem to be more "in focus" and sharp than the original, but not every player knows how to handle them.

Here are a few of the Gotchas that I ran into. I hope someone else finds them to be helpful.
  1. Don't use 127.0.0.1 as the IP address for the Master Backend server. It prevents you from connecting external MythTV boxes when you add a few to your setup. Type in the LAN IP for the computer. Even when I learned this, I wasn't able to change it in the settings and make it work. I ended up reinstalling. Try to get this one right the first time.
  2. If you want VNC, you don't need to manually install it. You can find it in the MythBuntu settings.
  3. The default option (for me) did not automatically update my schedule info. There's a check box in the setup wizards to "Automatically run mythfilldatabase" and this will fix it.
  4. If you want to stream tv with the built in flash player (like Youtube) make sure ffmpeg has the --enable-libmp3lame option, or your sound won't work. You can check this with the --version flag. I was able to get the right version by adding "deb http://packages.medibuntu.org/ hardy free non-free" to /etc/apt/sources.list and reinstalling ffmpeg (after apt-get update).
  5. Selecting the wrong "channel frequency" during setup caused my tuner to only have access to channels 2-13. Everything higher than 13 was static, as I found out the morning I tried to show the system off to my wife. The correct setting for me was us-cable.
  6. Don't bother enabling all themes: many are available with either wide-screen or 4:3. You don't need both.
  7. The package nuvexport allows MythTV to export to multiple formats. My Gotcha? It took 8 hours to convert 1 hour of video to Xvid with all of the quality settings turned up. I'll test with different options and post the data later.
  8. The Flash player in MythWeb doesn't (yet) know how to stream .nuv files, so don't transcode anything that you want to watch this way. It looks like later versions will fix this.
  9. Although the Flash player seems to be Flash 7, I wasn't able to watch it in Wii Opera browser.

Posted by Nesman | Permanent Link

Thu Jun 12 12:30:21 CDT 2008

MythTV - Introduction

MythTV is Linux's answer to Tivo. For the cave-dwellers among us, these are Personal Video Recorders. A PVR is basically what VCRs and DVD players want to be when they grow up. If you're familiar with Linux you know that Freedom is the central theme. With your Tivo, you can only do with what the service provider says you can do.

"Mother, may I keep a copy of this show on my backup server?"
"No."
"Mother, may I watch this show on my laptop, at my neighbor's house with him?"
"No."
"Mother, may I automatically remove the commercials from this before I watch it?"
"Hahahahaha... No."
"Mother, may I convert this show and watch it on my iPod or PSP?"
"I'm pretty sure that's piracy. You'd better buy a copy just for your PSP."

I hope you get my drift.

With MythTV, you can do all this and more. I won't even try to list all of the great features, but a few of my favorite are:
  • Commercial detection
  • Web interface for remote management and viewing
  • Distributed server setup, allowing for multiple recorders and viewing stations
  • Multiple network protocols, so you can connect any machine on your network
  • No subscription fees (Well, most of us pay $20/yr for listings, but there are free options)
  • Myth Game allows you to play roms on your TV. Hey, you bought the games when you were a kid; might as well keep playing them.
  • It's Linux! You can install it on your toaster if you can work out the drivers. I hear that heating elements only produce 5x1 pixel resolution, so you might want to try it on an actual computer.
Now, since I think there are enough reviews of MythTV out there, I feel obligated to provide some technical details. I'll try to make these available soon.

Posted by Nesman | Permanent Link

Sat May 31 22:21:17 CDT 2008

Todo.txt part 2: php interface.

Last week I talked about using a text file to manage tasks. To make it as easy as possible for my wife to use it, I modified a php script to use as an interface. Now, instead of opening a terminal we leave the page open in Firefox.

The page uses a form to take input, specialized links to mark items as done, displays my list, and can even send the grocery list to the printer. You can view the complete source code here: todo.php.

The page is a heavily modified version of code posted to the todo.txt Yahoo! group. The original is nice and compact, but lacks several features that I like. The real work of the script is interaction with todo.sh. This is handled by php exec. (See line 110 for an example) Everything else is child's play. The php page takes data from either _POST or _GET, depending on the action.

Most of the data is posted with the form, and usually for the grocery list. I added the default value "add @buy " to speed item entry. In Firefox, the page automatically starts the cursor just after this text so all you need to do is type. Although others tested browsers (Opera, Konqueror, and IE6) all place the cursor at the beginning of the text box, we generally use Firefox so it shouldn't be a problem. If you want to use the script for yourself but don't like the default text, change it on line 77.

Posted by Nesman | Permanent Link

Sat May 24 22:36:50 CDT 2008

Todo.txt - Using a text file to manage tasks.

As long as I can remember, I've used a text file for my todo list. I keep one at home that collects short-term goals like grocery shopping. Sometimes I use it for a scratchpad for URLs and random thoughts. I keep another one at work to remind me of hostnames, ip addresses, names, extensions, and anything else I need for a service call. Since switching to Linux, I've gotten away from having to keep my personal todo.txt on my thumbdrive. In fact, I've gotten away from using a text editor on it almost entirely.

Gina Trapani of Lifehacker maintains a small script to manage your todo list. Todotxt.org has the main bash script (todo.sh) along with several tools to integrate with it, including progress trackers, tab completion, and bots for Jabber and AIM. If bash isn't your language of choice, there is also a Python version. You can see a video demo of todo.sh on Youtube.

More than anything, I find myself using my todo.sh to manage a grocery list. Moriah and I leave it open on the computer so that we can add eggs or bread easily when we run low. To keep grocery items separate from "rake leaves" and "build workbench" we mark them with "@buy", which has the bonus of sorting them near the top of the list, all together. Then, we can view the list of groceries with this command:
$ todo.sh list @buy
23 @buy bread
22 @buy contact solution
19 @buy juice
20 @buy mayo


The cut[0] command, built into the GNU coreutils, makes it easy to grab just the important parts of that output.
$ todo.sh list @buy | cut -f3- -d" "
bread
contact solution
juice
mayo


Finally, I can pipe the output into lpr and print it.
$ todo.sh list @buy | cut -f3- -d\ | lpr

Next time, I'll go over the php script that I use as an interface to the shell script.

[0] Here, I use cut with two flags. -f3- means "from third field to end of line" and -d specifies the delimiter. Here I delimit with spaces, which you can quote or escape with a backslash.

Posted by Nesman | Permanent Link

Tue May 13 15:18:39 CDT 2008

Tried to Count in Base3 got 3-Sided Fractal

Recently, I got a wild hair in an uncomfortable place that suggested I look into trinary counting. Binary is great, but it doesn't seen very efficient. Each number holder is seen as either 0% or 100%. It would be nice to use a more precise system. Instead of splitting each digit in half, give 3 possibilities.

To test this, I tried to visualize how I might build a counting machine that worked in binary, but represented a base 3 system. I'm pretty sure that I got it wrong. My plan was to have a line of toggle switches that would flip (or not) based on the value of the previous switch. Each increment would toggle the switch on the right, so the next switch would only be flipped every other time. I wasn't sure, but I figured this might be a good place to start. My experiment in testing this had interesting results.

After working on it for a while, I began to picture it as a bucket brigade. The digit on the far right (Number 0) was the bucket brigadier in charge of the spigot. (We call him Number 0 because he doesn't know how to catch. This is why he's in charge of the spigot.) Each second, Number 0 fills his bucket and throws it to the next person (Number 1). If the next person has an empty bucket, he catches the water and turns to face the next person in line. The next second, everybody with water in their bucket throws some to the next person. They only throw half of the water, to make sure they never run out. (Don't worry about this part too much. Just run with it) They continue doing this until they are hit in the back of the head with water from the previous person, which causes them to turn around and prepare to catch the next bucket.

In this system, Number 0 will flip Number 1 each second. Number 2 (the next person) will catch a bucket of water (either in her own bucket or to the back of her head) every other second. This will cause Number 3 to flip every 3 seconds and so forth. In this way, the first bucket of water takes 1 second per person to make it to the end of the line. Following buckets are more and more likely to end up being wasted, but it makes for an interesting counting experiment.

To help me visualize this, I would need a script to calculate the state of each person in line, each second. The main mechanic is that each digit is flipped only if the previous is in the "on" position (throwing water). Eventually I figured that I could represent each state as 1 or -1. To determine what happens to any particular digit, just multiply it by the one on it's right. Here, we consider 1 to be "off" because multiplying causes no change, while -1 is "on" because it will toggle negative status. Unfortunately, 1 and -1 are not very pretty in large groups; they just look like vertical and horizontal dashes.

Before I could go much further, I had to figure out how to do my calculations with 1 and -1 while displaying something that looked decent. I ended up with two arrays: all of my heavy lifting was done in the first, then the pattern was copied to the second as a prettier version. My first thought was to use an if/then selection to swap the value: if $val == -1 { then $binval = 0 }; That just felt lazy. I never tested it, but I'm sure that's two slow. Finally I found a method to convert either value to 1 or 0 from 1 or -1: $binvar = (($var +1) / 2); This will use -1 to toggle, but display 0 for better contrast.

Finally, it's time to run it and watch my numbers run. An early test looked like this: (each second is displayed below the previous, and counting starts from the right)
	1 1 1 1 1 1 0
	1 1 1 1 1 0 1
	1 1 1 1 0 0 0
	1 1 1 0 1 1 1
	1 1 0 0 1 1 0
	1 0 1 0 1 0 1
	0 0 0 0 0 0 0
	1 1 1 1 1 1 1
Remember that here, 1 is off, 0 is on for the ability to toggle the number next to you. Also, the number on the right is toggled for each new row. This is OK to do manually for small numbers, but to really test the concept, I wanted to run it with hundreds of columns and rows at once. Since 1's and 0's take up more room than needed, I loaded the result into opera and zoomed out. Here is a screen shot of my result:
Sierpinski Triangle in ones and zeros

My result was a kind of tilted Sierpinski Triangle. Although I completely failed to count in base 3, I succeeded in building a 3-sided fractal.

The final version of my code can be found here: negcount.php You should be able to run this on any web server with php4 if you would like to play with it.

Update:After a little research, I see others with code for the same thing. Jason Underdown wrote Sierpinski's Triangle in C which is 3 times as long as mine, if you count comments and twice as long without comments. (Found via Wikipedia article for Sierpinski Triangle).

Before I brag too much, I should point out this example (done in Ruby) of a one-liner Sierpinski's Triangle:
ruby -le '64.times{|y|print" "*(63-y),(0..y).map{|x|~y&x>0?" .":" A"}}'

Posted by Nesman | Permanent Link

Wed Apr 9 12:44:46 CDT 2008

btdownload: Command-line Bittorrent client (in Python)

Bittorrent is a great way to get Linux iso's. Not only can you get a fast download without killing the server of your favorite distro, but your bandwidth can be used to help distribute the file to others! The downside here is how easy it is to max your outgoing bandwidth, choking any outgoing services that you might run. On top of killing your personal website, browsing the internet from this machine will be painful. When your upload is maxed out, it can severely limit your ability to download web pages and even your torrents.

My favorite torrent client is a Python script called btdownloadcurses.py . It has several incarnations but I prefer the curses version. I choose the command line client over a GUI because it can be very painful to make a remote desktop connection while you download. With btdownloadcurses running in a screen session it is very easy to ssh home and check my progress.
screen -S bt #start a screen session named "bt"
btdownloadcurses --max_upload_rate 45 /path/to/file.torrent #My real max upload is around 55KB/s, so I can give 45 to my torrent.
^a d #"<ctrl>+a, d" detaches my screen session.


Another useful trick: when you want to download more than one torrent at a time, you can run them individually in different screen sessions or windows ("<ctrl>+a, c" creates a new window, then "<ctrl>+a, [p|n]" to select previous/next window in screen). Alternatively, you can use btlaunchmanycurses.py to start all the torrent files in a directory.

Two things to note with btlaunchmany:
  • Max upload will need to be calculated per file: if you are willing to put 45KB/s into uploads and have 3 torrents in the directory, you'll want to set max_upload to 15.
  • The normal client takes options before path, but btlaunchmany takes path first.
btlaunchmanycurses /path/to/torrents/ --max_upload_rate 15

This is a great way to seed several torrents that you've already finished downloading.

Posted by Nesman | Permanent Link

Mon Mar 17 15:16:10 CDT 2008

Using ls or du to check progress on remote bittorrent download.

your first impulse might be to make a GUI connection with VNC or some other remote desktop software. If you've ever tried this, you'll remember how painfully slow the connection can be when your torrent has your connection maxed out. After you authenticate, you have to see the screensaver for a second before it fades away, click to the right desktop, find the right window, and finally you can see your progress. Each step takes several seconds while you wait for the screen to load. Not to mention the fact that you're chewing into your bandwidth and hurting your transfer.

The better way to do this: ssh. With ssh, you use almost no bandwidth. You can easily navigate to your download directory and check file size in a few seconds. Which tool you use will depend on the way your bittorrent client creates the file. Some clients reserve space for the entire file when you start while others build the file as data comes in.

If your client reserves the space when the download starts, the ls command will say you have the entire file. The only clue you'll have is file date: it will continually update even though the size stays the same. To see how much space is actually being used on the disk, use the du command.

For example, du -sh ~/OperaDownloads/*
This code will list the size of each file and folder in OperaDownloads.

You can test the difference with a sparse file. You can easily create one with this command:
dd if=/dev/zero bs=1k count=1 seek=`expr 1024 \* 4096` of=~/sparse.file
You will now have a 4GB "empty" file. file calls it data, ls -lh calls it 4gb, but du and df know the truth. Just for fun, you can tar.gz your sparse file and see just how small it is: tar -czf sparse.tgz sparse.file

Posted by Nesman | Permanent Link

Mon Feb 11 20:40:01 CST 2008

Math in monitor resolutions

Most people never think about it, but there's a bit of math that goes into your options for a monitor resolution. Standard monitors use a 4:3 aspect ratio. This means that for every 4 inches wide, it will be 3 inches tall. Widescreen monitors are usually 16:9. Further reading: Aspect Ratio on Wikipedia

If you wanted to measure dpi, you might use the Pythagorean Theorem to find how many pixels your monitor has from corner to corner and divide this by the number of inches. (Diagonal inches are the measurement used to sell monitors.) About 90 dpi is average: while a higher dpi produces a more crisp image, a smaller number is easier on your eyes. Normally, the smallest standard resolution is 640x480, followed by 800x600 then 1024x768. These have been common for quite a while and most computers can manage much more these days, but 1024x768 on a 17" monitor is still standard.

Have you ever wondered where the numbers we use for standard resolutions come from? I could be wrong, but if you figured out the dpi for a 640x480 screen, you may have noticed that the diagonal measurement was 800 pixels.
sqrt(640^2 + 480^2) = 800
It seems that in A^2 + B^2 = C^2, our old C becomes the new A. This doesn't continue, sadly. You would expect the next resolution to be 1000x750, but 1024x768 is prettier in binary. (10000000000x1100000000) It also makes better use of video memory address space, but I'm sure that's a side effect. Besides, 1024 is pretty much the same as 1000 anyway, right? That's what hard drive manufacturers have been telling us for years.

If we continue, we'll see that the next logical resolution would be 1280x960, but most systems skip that and go to 1280x1024, giving you a more square 5:4 ratio. A square monitor would give you more viewing space per diagonal inch, but if you want to watch anything in widescreen format it doesn't help at all. Instead, you get thicker black bars at the top and bottom of your screen.

Speaking of widescreen, the standard format for movies tends to be 16:9. If you were playing around with figuring out the diagonals, you might have noticed that our 4:3 makes a textbook Pythagorean triangle: 4x3 with a diagonal of 5. While you were doing the math you might have noticed that your A^2 and B^2 came out to 16 and 9. That's right. The widescreen aspect ratio can be found by squaring the numbers in the standard, square ratio.


Posted by Nesman | Permanent Link