»In which we have fun with Disk Futility
Thanks to a tip from Greg, I set my Crash Reporter preferences to "Developer", and was able to attach a debugger to a crashing Safari:
Exception: EXC_BAD_ACCESS (0x0001)
Codes: KERN_INVALID_ADDRESS (0x0001) at 0x017fe7dc
Reading symbols for shared libraries ............. done
/Users/salim/341: No such file or directory.
Attaching to program: `/Applications/Safari.app/Contents/MacOS/Safari', process 341.
Reading symbols for shared libraries .................................................................................
warning: Can't find LC_SEGMENT.__DATA.__data section in symbol file
.................... done
0x901d134a in TAATAcceleratedMorph::SetupTableData ()
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x017fe7dc
0x901d134a in TAATAcceleratedMorph::SetupTableData ()
(gdb) bt
This suggested a memory problem, which could be the stuck i/o that I saw earlier. Processes would remain in the process table forever, with the kernel waiting to complete some sort of network, disk, or, as it turns out, memory, operation. I opened up the OS X Disk Utility, and found hundreds of problems when I asked it to Verify Permissions on the startup disk, /. (Frustratingly, the Verify and Repair Permissions are separate buttons but produce similar output, concluding with "The privileges have been verified or repaired on the selected volume." Several passes later, many things seem to be working well. One curious artefact: the disk-repair effort revealed that the /usr/standalone/i386/boot.efi file had read-only permissions (fixed) and that it contains the stern admonition "This program cannot be run in DOS mode."