AgeStar CFB3AT FireWire Enclosure problems in Solaris
I recently needed to add a new external disk to my SunBlade 2000 I had a spare 500GB SATA drive laying around and I found a drive enclosure that supports both PATA and SATA drives and has USB 1.0, 2.0, Firewire 400 and E-SATA connectors on it. at http://www.cooldrives.com/esu sb20andfi.html
I had nothing but trouble with it under Solaris 10/SPARC but it ran perfectly
fine on Linux, Windows and MacOS X.
I would get the following errors when copying lots of data to this drive:
scsi: WARNING: /pci@8,700000/firewire@5,2/unit@0050770e00071002/disk@0,0 (sd37):
Oct 11 22:27:04 ayeka SCSI transport failed: reason 'reset': retrying command
Oct 11 22:27:05 ayeka transport rejected fatal error
Oct 11 22:27:05 ayeka ufs: WARNING: Error writing ufs log
Oct 11 22:27:05 ayeka ufs: WARNING: ufs log for /mnt changed state to Error
Oct 11 22:27:05 ayeka ufs: WARNING: Please umount(1M) /mnt and run fsck(1M)
I did a ton of fiddling around and finally I have success! I had to dig around the opensolaris.org code browser looking at the scsa1394 driver. There is an undocumented driver variable called scsa1394_wrka_fake_rmb
Don't ask me what it does exactly all I know is it is off by default and setting it to 1 instead of 0 seems to have fixed my issue. Or at least somewhat anyway. I had it cloning my LaCie BigDisk for most of last night and it did eventually die. I'm going to try re-enabling the symbios workaround and see if that fixes it completely. In the mean time I'm using rsync to do the copy and it seems to be reliable enough to replace my ailing LaCie drive. If things continue to be a big PITA I will just install a USB 2.0 card in my Sun and be done with it.
This tip should work with any enclosure that uses a Prolific Technology Inc. PL-3507 chipset but YMMV. I hope someone finds this info useful because I sure wish I had found it before having to get elbow deep into the guts of Solaris out of frustration.
To set this yourself try putting the following lines into /etc/system and rebooting. I'm not sure that scsa1394_wrka_symbios = 0 is needed.
*Try some FireWire system tuneables
set scsa1394:scsa1394_wrka_symbios = 0
set scsa1394:scsa1394_wrka_fake_rmb = 1
For more information on using firewire drives with SPARC machines take a look at This article
[/solaris/firewire] permanent link RSS feed
Open Pandora an idea who's time has come
I just stumbled across the Open Pandora Project and I am practically drooling over this awesome wonder of a device. It is unfortunate that it is primarily focused on gaming because this little gem is capable of so much more!
This little device fits in your pocket, has blue tooth and 802.11 wireless technologies, a battery life around 10 hours. two SDHC slots, two USB ports, an S-video out etc. In short it is the pocket sized general purpose computer (Super PDA if you will) that I always dreamed of ever since those Casio organizers came out. (I always wanted a Casio organizer that could run DOS or UNIX back in the early 90s)
If the first batch starts shipping and the orders keep coming in for this device I am sure to scrape up the cash to get one. This is the perfect device to run TN and a bunch of other applications that are perfect for a pocket sized device like this.
I still plan to get a nice ARM based netbook as well but the awesome possibilities of this device can not be under-stated. It is easily as powerful (actually a lot more powerful) than an OLPC XO-1. and can use blue tooth keyboards etc. It has the same 256MB ram limitation as the XO-1 but if you know what you are doing you can still cram a crap load into that considering that in the 90s I had useful unix workstations running on much less than that.
I believe that powerful, open and compact general purpose devices like these that are completely open source are the future. versatile mobile computing devices and fresh new ways that we will be looking at mobile networks will change the world.
[/projects/teotwawki] permanent link RSS feed
I haven't posted anything on TEOTWAWKI Net in a while so here's an update:
I've been working on the USB sync manager which is simply a script that looks for mount points being added in a certain directory. it then scans that mountpoint for spooled up batch files containing news articles.
Once it finds the batched up articles it reads them in and deposits it's own articles onto the mounted device. once it is done processing all found mount points it will pop up a GUI (optional) listing all found mount points with an unmount button next to each one. This menu will disappear as soon as it starts processing again. The menu will not pop up if there are no more mount points left in the directory it looks in.
USB keys left plugged in will accumulate any new articles that arrive either by the network or by other USB keys being plugged in.
What I just wrote about is not just an idea but is already working now and is in CVS. I just got a lot of this working over the weekend and some more tonight.
I plan on making some changes to how the system is packaged and splitting some things out of start.sh and writing some new documentation to put in the webroot before I build more packages. At that point I might have my first numbered release. Stay tuned.
[/projects/teotwawki] permanent link RSS feed
Had fun repackaging OpenDX for Slackware today
I spent about 3 hours re-packaging OpenDX to run on Slackware for a customer today. It was nice to just get back to some good old fashioned UNIX admin work for a change.
Some people may hate the Slackware package management system or accuse it of just being a tar file (or being nonexistent) but they would only be partly right.
In Slackware package files are just tar files but the real brains behind the packaging is inside the package utilities themselves. The packaging utilities are wrappers around tar and maintain a catalog of files installed with a package so that it can be removed or upgraded later. They also can execute shell scripts included in the tarball inside of the install directory. There is no dependency tracking but the same can mostly be said of System V Release 4 packages (as used on Solaris)
Either way love it or hate it the Slackware package system is very easy to work with as a result of it's simplicity. build your project into a directory tree that reflects how the files should look relative to the root file system. put any extra needed stuff inside the install directory and build it with makepkg and you are done.
You can even use rpm2tgz to transfer an rpm into a regular tar file and work off of that if you have to. or just build the tar yourself without makepkg and still get the benefit of being able to uninstall it using removepkg later on.
I subscribe to the idea that small and simple is beautiful. Slackware packages may be lacking in modern features but they are simple and easy to work with which I find quite refreshing in this age of horribly complex software.
[/slackware] permanent link RSS feed
I found a fascinating open source hardware and software project called Arduino that has a surprising amount of sophistication and peripherals available for it. and a seemingly easy to use software development environment.
One of the options is to hook it up to blue tooth or 802.11 wireless modules. I'm thinking that I might be able to take the 802.11 or blue tooth modules (802.11 module supports TCP/IP so it is the obvious choice) and if an SD card can also be utilized I might be able to implement a cheap hardware device that can be placed in your pocket and gather/propagate TEOTWAWKI Net completely unattended. you could then sync it with your laptop or other larger system later.
I have no idea if the Arduino is capable of anything as complex as TN but it was enough to get me thinking. Eventually I would like to implement the network on smart phones and sophisticated PDA devices but open source hardware platforms like this really open a lot of cool possibilities.
I just spent a couple of hours looking at information about the platform and the microcontroller chip it is based on and it may be a bit too simple to do what I want. however It is still quite capable and flexible for small projects. I really want to see a lot more work being done in the open source hardware area. the amount of new and innovative hardware and software projects that could be built really is unlimited with completely open hardware to build on.
What i would really like to see is a project very much like Arduino but targeted at much larger scale applications. Something that is about the same size as the Arduino and has the ability to plug in multiple I/O boards with storage, wifi, etc. as well as exporting various pins you would find on a micro controller. It would also need to be able to run a decent real time OS (hopefully something POSIX like or even a small version of Linux). I could do a lot with a device like that.
In a nutshell what I would really like to see the hardware hackers come up with is a low cost general purpose embedded computer platform that is small enough to fit in someones pocket and run off of batteries. With enough CPU horsepower and features to run complex wireless network applications. I don't even need a display or a keypad but a modular system would make this easy to add. If the open source hardware hackers ever come up with such a thing I will be very happy.
I'm still tempted to buy some of these kits and see just how far I can push things though. (If only I had unlimited time!)
Ohhhh Yeahhhh! check this out: http://blogs.leaflabs.com/2009/
This looks like it may well just fit the bill for what I'm wishing for I will be following the development of this project very closely. My main focus is to get the UNIX version of TN working as well as I possibly can before I go off on porting to other platforms embedded or otherwise. but once that goal is accomplished I will be getting some kits and exploring the possibilities.
[/projects/teotwawki] permanent link RSS feed
New packages released for the LUGOR/OLPC picnic this Saturday
The packages are built from the code with the lugorpicnic tag in CVS this version includes the new and improved network mapper and sync dispatcher and an important bugfix to the syncnews.pl script (I got the article date and article arrival time backwards when reading the history file) Also as a consequence of the IPC provided by the new network mapper. the network neighborhood is now implemented as a PHP script that reads the network map from the daemon and works much better now.
The new IPC mechanism also allows the syncnews.pl script to update a "lastsync" timestamp in the network map when it successfully syncs with a node. This fixes another problem I had with the old mapper script that could cause some articles to never be sent out to other nodes.
This release also includes the incredibly useful socat utility which can be used to talk to the network mapper socket and for just about anything networking related you can think of.
[/projects/teotwawki] permanent link RSS feed
The new network mapper/sync dispatcher has just been committed to CVS. It implements everything I talked about in the previous blog post. the following are the implemented commands for the /tmp/TNmapper IPC socket.
all commands close the socket when they are finished.
UPDATE will return ACK\n on success and NACK\n on fail.
NTOA will return the ip address followed by \n or NACK\n if the nodename is not in the n2a hash.
ATON will return the nodename followed by a \n or NACK\n if the ip address is not in the map hash.
DUMP will return multiple lines containing the entire network map. it is comma separated into the following fields ipaddr, nodename, services Once it is finished dumping the socket is closed. it can return zero lines for an empty network map.
DEBUG This will output the %map and %n2a hashes to the IPC socket in Data::Dumper format.
[/projects/teotwawki] permanent link RSS feed
Looks like I have to write a comment system.
This is partly my fault because I either don't understand something or I just had to pick the most obscure blogging platform out there. But I have yet to find a comment plugin for blosxom that actually works or has understandable documentation. All of the plugins I have found are either totally undocumented or woefully out of date.
So I'm just going to have to roll my own comments system You will just have to wait until I get around to it. (I have some general ideas on how to get this done)
I normally wouldn't bother but I rather like the way blosxom does things and wordpress is a giant bloated mess. I just want to get back to simple software with a small footprint modern software is driving me crazy and I'm just sick of it.
[/rants] permanent link RSS feed
A new and improved Network mapper/Sync dispatcher
I've been busy learning a lot more about sockets programming from W. Richard Steven's UNIX Network Programming 1st edition. and applying some of that info to perl. I am modifying an example server that uses select and non blocking I/O to eventually be the new network mapper and sync dispatcher. I already have this test server accepting both UDP connections and AF_UNIX (local system only accessed by unix filesystem paths, (/tmp/afunixsocket for example)) streaming connections and passing them to different handlers.
The end result will be that other processes will be able to communicate with the heart of the TN system and get information about the network and be able to update the crucial time stamp information for each node (the syncnews.pl script needs to be able to do this since it is spawned into the background.)
The reason I'm focusing on this change is that the original script for the network mapper and sync dispatcher just does not seem to be easily extendable the way it currently exists and some of my testing has brought up certain design problems that can cause articles to be lost and the only way forward will be to have the mapper support IPC and add more information to the network map that can be accessed and updated by processes that need to know about and work with this information.
I have wanted to do this from the beginning anyway with the network neighborhood. it is much better to make that a script that gets info from the network map directly. instead of an ugly hack where the mapper daemon re-writes a static html file when it updates the network map.
The new way the process will work will also allow things like a name service switch plugin to be developed that will allow hostname lookups to use the TN network map.
I'm also going to make it a bit of a hybrid server in that while it does select and non-blocking I/O instead of forking. it will only do so for receiving the UDP broadcasts that build the map and handling commands from the AF_UNIX socket that update the map. any other commands will cause it to fork a process that will block and process all other commands namely dumping of the network map and doing specific map lookups. that way the main daemon will not have it's time taken up by queries about the map and since that is handled in a child process that child will only have a copy of the network map at the time of the fork. I also found a really awesome tool that blows away netcat called socat that I think I will include in the TN system for debugging and possibly other purposes later. not only does it allow me to interactively send data to and read from AF_UNIX sockets but it is an incredible networking swiss army knife that allows all sorts of different possible socket combinations to talk to each other.
[/projects/teotwawki] permanent link RSS feed
I have dug out xfig and made some simple figures showing how TEOTWAWKI Net works.
the arrows pointing to the ad-hoc cloud are as follows:
[/projects/teotwawki] permanent link RSS feed
Might have found a way to make an XO act as an AP
I did some digging around and asking on the olpc-devel irc channel and it looks like there is a driver and special "thin firmware" for the libertas chipset that supports AP mode. The guy that maintains the DebXO distro is having me test out a package that includes this new firmware. I will let you and him know about it as soon as I get a chance to test it. theres also instructions for getting the thin firmware working on the stock sugar, but it requires re-building the kernel with the correct options set.
(I will probably create a package for this if it all works out nicely under debian)
It will be so cool to be able to put an XO into AP mode. I've been making further improvements to the sync-up code and am working out some more bugs and timing issues. I also got my UNIX Network Programming 1st edition in the mail today as well.
What a wonderful history book to read from back in early 1990 when most of the world was not POSIX compliant, there were several competing network protocols with TCP/IP and some UNIX vendors were just getting around to getting ANSI compliant compilers (a lot of the examples are K&R style C but the sockets interface really hasn't changed much at all)
It deals a whole lot with interprocess communication and has some rally sophisticated examples and is very in depth. I know I'm going to learn a lot from it to make my project better.
[/projects/teotwawki] permanent link RSS feed
