Sorcerer/Micropolis Disk Imaging with FluxEngine

My Sorcerer uses a Micropolis hard sectored floppy drive system. It uses 16 sectors, each of which includes a 256 byte data section and 19 bytes of bonus “metadata” used by Micropolis DOS (MDOS). This metadata is ignored by the CP/M implementations (Exidy & Software Source) that i have encountered.

The drives have an unusual track spacing of 100TPI. This is unusual so imaging has to be done using the actual drive units.

FluxEngine is a very low cost imaging solution that support Micropolis hard sector disks as used on my Sorcerer. At the time, Greaseweazle did not support hard sectors. It has since been added. I’ll do a separate post on that.

I followed the instructions here:

https://cowlark.com/fluxengine/doc/building.html

I connected to the drive using a straight through cable and using drive:0.  This selects the second drive in the dual drive unit (drive 1).  For the single drive unit, DS1 (the second drive select) must be selected.

I used FluxEngine on a new PC, so there would be no USB bandwidth issues.  My older machine does seem to work reliably, but if it gets busy then FluxEngine will sometimes throw a bandwidth error.

Initially i used my single drive unit (which I call Drive 2) mainly because it was compact. Once I started working my way through the disks, I found I was getting lots of errors.  I thought that I might do better with the dual drive unit (I call the top drive “Drive 0” and the bottom drive, “Drive 1”).  I found that Drive 0 was a little better than Drive 2 and that Drive 1 was the best drive to use. 

The results were a little disappointing but some further reading of the FluxEngine doco:

http://cowlark.com/fluxengine/doc/problems.html

hinted that I could fiddle with decoder.bit_error_threshold to perhaps get better results.  0.2 is the default.  0.4 improved things a lot.  0.5 was the best.  With the wrong setting, a lot of empty (?) sectors were read.  I also increased the retry limit.

Drive 1 remained the best performing drive.  I tried messing with the alignment on Drive 2, but I could not noticeably improve its performance.  I did some poking around with the scope but no obvious differences between the dual and single units leapt out at me.

Out of 87 disks, I was able to read 19 without any errors being reported. I was able to read another 18 with only a small number of errors.  A further 38 disks didn’t yield anything usable.  Some were shedding oxide, but most 24 were successfully reformatted.  Many were unlabelled, so perhaps no great loss.

fluxengine read micropolis --315 --layout.tracks=77 --layout.sides=1 -s drive:0 --decoder.retries=8 --decoder.bit_error_threshold=0.5 -o exidy_%1.img

Disks that were formatted with MDOS were read using the 275byte sector size.  CP/M disks were read with 256 byte sector size.  My understanding is that CP/M doesn’t use the “metadata”, and the 256 byte size is much easier to view.

fluxengine read micropolis --315 --vgi --layout.tracks=77 --layout.sides=1 -s drive:0 --decoder.retries=8 --decoder.bit_error_threshold=0.5 --decoder.micropolis.sector_output_size=275 -o exidy_%1.img

There were some examples of where I was able to copy a disk on the real machine, but FluxEngine was unable to read the disk without error, eg Disk 24.  The copy read fine.  This meant that there was potential to improve the yield by making fresh copies on the real machine.  Ie The real disk controller is still doing a better job in real time than the flux engine does in slow time.  That’s the advantage of inside knowledge, perhaps.

I attempted to copy all the errant disks to intermediate disks, which I subsequently imaged.  I did this for the CP/M disks using COPYDISK.  This yielded another 13 good images!  I think this confirms that FluxEngine is way less tolerant of problems than the sorcerer system.  Even many disks that had a lot of issues came good.

I repeated the exercise for the MDOS disks, which gave another 8 good images.

I was able to write back both CP/M and MDOS images:

fluxengine write micropolis --315 --layout.tracks=77 --layout.sides=1 --decoder.retries=8 --decoder.bit_error_threshold=0.5 -i exidy_%1.img -d drive:0

fluxengine write micropolis --315 --vgi --layout.tracks=77 --layout.sides=1 --decoder.retries=8 --decoder.bit_error_threshold=0.5 -i exidy_%1.img -d drive:0

The write process verifies each track by reading back so even the writes need the bit_error_threshold parameter.

Writing also work through the virtual sector generator to a soft sector disk.  The disk can be read by the system if the virtual sector generator is used.

I was very impressed with FluxEngine. David Given has done an amazing job.

Using Greaseweazle with Sorcerer/Micropolis Disks

The Micropolis disk system uses hard sectored disks. This is particularly challenging for disk imaging systems.

Release 1.20 of Greaseweazle included hard sector support, so i gave it a try. I was able to read and write disk images with moderate success. Without a codec for the disk formats the software could not check each track and retry if necessary.

The Greaseweazle developer, Keir Fraser offered to add the fluxengine codec into the code which he did very rapidly.

I tried it out and found it worked very well.

I did find that some of the default parameters didn’t work well with my drives, but these were changed with the delays option.

The relevant commands are:

Get helpgw -h
Speed testgw rpm –-drive=0
Head cleangw clean –-drive=0
Set some safe drive parametersgw delays –-step=30000 –-settle=10
Reads a micropolis disk into an scp file.  The raw option keeps the sector pulses and allows reprocessing.  Without the raw option the files are more easily viewed with the HxC tools.gw read –hard-sectors –-drive=0 –revs=5 –tracks=”c=0-76:h=0″ –-raw %1.scp
Writes a micropolis scp file to a diskgw write –hard-sectors –-drive=0 –tracks=”c=0-76:h=0″ –-raw %1.scp
Reads a SS 100TPI 256 byte sector micropolis disk and decodes it into an IMG file.gw read –hard-sectors –drive=0 –format=micropolis.100tpi.ss –retries=8 %1.img
Writes a SS 100TPI 256 byte sector micropolis IMG file to a diskgw write –hard-sectors –drive=0 –format=micropolis.100tpi.ss –retries=8 %1.img
Reads a SS 100TPI 275 byte sector micropolis disk and decodes it into an IMG file.gw read –hard-sectors –drive=0 –format=micropolis.100tpi.ss.275 –retries=8 %1.img
Writes a SS 100TPI 275 byte sector micropolis IMG file to a diskgw write –hard-sectors –drive=0 –format=micropolis.100tpi.ss.275 –retries=8 %1.img

CP/M files are read with the 256 byte option format=micropolis.100tpi.ss

When creating the image files, the sector integrity is checked and if the sector is bad it is retried. It was common for multiple retries to be required.

I did try to adjust some parameters (i had to with fluxengine) but i could not get much improvement. Nevertheless, it was quite satisfactory.

The actual system does better, so it often helps to make a fresh copy of a disk on the Sorcerer before trying to image with either FluxEngine or Greaseweazle.

I also tried using Greaseweazle to write micropolis images to soft sectored disks with a Virtual Sector Generator. It also works fine.

What i have not been able to do, so far, is to trick the Sorcerer into using a Gotek/Flashfloppy either with or without the Virtual Sector Generator.

As at V1.22 Greaseweazle includes the micropolis codec.

I use Greaseweazle a lot so i really appreciate the effort that Keir and others have put into its development.