Note: this documentation was written on 2004-02-05. Quite old, but it might help someone with such a machine (they are true wonders, I would still consider buying one nowadays). The links might be broken, too.
Actually, the most difficult part in installing Linux on this machine is booting. Apparently the only way is to boot from an usb floppy disk and then make a network install, or boot from the provided pcmcia cdrom device. As I the cdrom at hand, and both a Knoppix v3.3 and a Debian woody 3.0r2 cds, I decided to try the pcmcia cdrom boot method. Now, for the subtleties:
I attempted to boot the knoppix cd, but there were read errors when it tried to install some basic software. So I booted the woody cd, and it worked like a charm!
I installed the Debian distribution as usual, it isn't much different from usual debian installation.
As I wanted the kernel to support the latest drivers, I decided to recompile a linux 2.4.24 kernel on another machine, burned it on a cd (the usb key I had couldn't hold enough), and installed on the sony. See the config file provided here.
There is a kernel patch known to make the console use all the width of the screen, but it seems to work well only for the 2.4.17 (see the Links).