It's easiest to prepare video for the IPod Nano just by using the Handbrake software in command line mode. The GUI interface requires Net 2.0 and it seems a bit buggy in a virtual pc.
The following seems to have smaller video size:
HandBrakeCLI.exe -i DVD\MYVIDEO\VIDEO_TS -o myvideo.mp4 --preset="iPhone & iPod Touch"
Other option:
HandBrakeCLI.exe -i DVD\MYVIDEO\VIDEO_TS -o myvideo.mp4 --preset="iPod"
Encrypted DVD's Finding Title
The following command will show all of the menus, find the right title, and then use that with the -t switch to set it, default is always 1.
handbrake -i /dev/dvd -t 0