Note: this post doesn’t provide a solution to address the “Incorrect number of thread records” file system issue but documents what I went through to see if I could when I faced it. I think your best bet would probably be to try ALSOFT’s Disk Warrior if you need to fix this issue.
I was sitting in front of a Mac Pro which had previously suffered from boot issues due to the expansion boards holding the memory not making contact with the logic board properly. The boot issue manifested itself as the system not POSTing and the fans spinning loudly. Luckily the issue was resolved by reseating things. The system also had a failing Cinema Display which would work fine after a “cold” power cycle having removed the power connector from PSU and reattached, but would fail to wake from sleep if it had been left on. The user was not aware of this issue and assumed whenever the system wouldn’t display anything on boot, it was the POST issue and would power cycle the system, hoping to strike it lucky. Through doing this, the HFS+ journaled file system ended up with problems where fsck_hfs(8) was unable to fix the issue.
Since the filesystem had problems, the system would try to boot, the grey screen with the Apple logo would appear along with a progress bar which would move from left to right and then after some time the system would switch off.
Booting the system in verbose mode by holding command + v showed that the system was running fsck on boot which is what the progress bar was an indicator for normally. It would attempt to repair the file system 3 times, fail, and switch off.
Connecting to another Mac via TDM, it was possible to mount the filesystem as read only and take a closer look. It would have been possible to also boot the system in rescue mode but the Mac I connected to via TDM was running a newer version of macOS and I was hoping that the tooling would be better able to deal with such an issue though fsck_hfs(8) bugs section states “fsck_hfs is not able to fix some inconsistencies that it detects“, regardless.
% sudo fsck_hfs -ypd /dev/disk2s2
/dev/rdisk2s2: starting
journal_replay(/dev/disk2s2) returned 0
Using cacheBlockSize=32K cacheTotalBlock=32768 cacheSize=1048576K.
Executing fsck_hfs (version hfs-522.100.5).
** Checking Journaled HFS Plus volume.
The volume name is Macintosh HD
** Checking extents overflow file.
** Checking catalog file.
Incorrect number of thread records
(4, 21962)
CheckCatalogBTree: fileCount = 421327, fileThread = 421326
** Checking multi-linked files.
Orphaned open unlinked file temp2639983
Orphaned open unlinked file temp2651454
Whole bunch of these temp files trimmed from listing
** Checking catalog hierarchy.
** Checking extended attributes file.
** Checking volume bitmap.
** Checking volume information.
Verify Status: VIStat = 0x0000, ABTStat = 0x0000 EBTStat = 0x0000
CBTStat = 0x0800 CatStat = 0x00000000
** Repairing volume.
FixOrphanedFiles: nodeName for id=2671681 do not match
FixOrphanedFiles: Created thread record for id=2671681 (err=0)
FixOrphanedFiles: nodeName for id=2671681 do not match
FixOrphanedFiles: Created thread record for id=2671681 (err=0)
FixOrphanedFiles: nodeName for id=2671681 do not match
FixOrphanedFiles: Created thread record for id=2671681 (err=0)
FixOrphanedFiles: nodeName for id=2671681 do not match
FixOrphanedFiles: Created thread record for id=2671681 (err=0)
** Rechecking volume.
Repeat again a second time
** Checking Journaled HFS Plus volume.
The volume name is Macintosh HD
** Checking extents overflow file.
** Checking catalog file.
Incorrect number of thread records
(4, 21962)
CheckCatalogBTree: fileCount = 421327, fileThread = 421326
** Checking multi-linked files.
Orphaned open unlinked file temp2639983
Orphaned open unlinked file temp2651454
Whole bunch of these temp files trimmed from listing
** Checking catalog hierarchy.
** Checking extended attributes file.
** Checking volume bitmap.
** Checking volume information.
Verify Status: VIStat = 0x0000, ABTStat = 0x0000 EBTStat = 0x0000
CBTStat = 0x0800 CatStat = 0x00000000
** The volume Macintosh HD could not be repaired after 3 attempts.
volume type is pure HFS+
primary MDB is at block 0 0x00
alternate MDB is at block 0 0x00
primary VHB is at block 2 0x02
alternate VHB is at block 975093950 0x3a1ec0be
sector size = 512 0x200
VolumeObject flags = 0x07
total sectors for volume = 975093952 0x3a1ec0c0
total sectors for embedded volume = 0 0x00
CheckHFS returned 8, fsmodified = 1It was possible to find out the offending files by looking up the inodes listed in the fsck_hfs output. Note the id reported in “FixOrphanedFiles: nodeName for id=2671681 do not match” and use find(1) to look it up
% find /tmp/p -inum 2671681
/tmp/p/private/var/audit/20210215092939.crash_recovery
/tmp/p/private/var/audit/20210215093035.not_terminatedIronically, the files causing the issue are auditd(8) logs from crash reports?
I thought perhaps turning off journaling would help sidestep the issue by causing fsck to remove the offending files, rather than trying to make use of journal data to replay. hfs.util(8) which is tucked away in /System/Library/Filesystems/hfs.fs/Contents/Resources let’s you do that.
% sudo /System/Library/Filesystems/hfs.fs/Contents/Resources/hfs.util -N /dev/disk2s2
Turned off the journaling bit for /dev/disk2s2It didn’t help.
hfs.util‘s supposedly supports a -M (Force Mount) but I was unable to get this to work. I was hoping to force mount the file system read/write & delete the 2 files.
I ended up wiping the disk and reinstalling macOS.
As an aside, the history section of the hfs.util(8) claims it was “Derived from the Openstep Workspace Manager file system utility programs“. The sources for hfs v106 package on Apple’s site shed some more light. Oldest entry in “change history” section of hfsutil_main.c states
13-Jan-1998 jwc first cut (derived from old NextStep macfs.util code and cdrom.util code).
Note the description of what main() does in a comment block inside hfsutil_main.c.
Purpose - This our main entry point to this utility. We get called by the WorkSpace. See ParseArgs for detail info on input arguments. Input - argc - the number of arguments in argv. argv - array of arguments. Output - returns FSUR_IO_SUCCESS if OK else one of the other FSUR_xyz errors in loadable_fs.h.
There are icons for HFS formatted disks from Rhapsody in the directory: hfs_RHD.fs.tiff, hfs_RHD.openfs.tiff. These live on in hfs v556.60.1 which is the most recent version available on the site as I write this.