RetroChallenge 2013WW Review

This has been a fun month. I am really excited that I completed one project and hopefully provided a useful tool to the Apple II community.

Disk ][ Emulator: I just uploaded the USB AII-SPI-DSK executable and schematic. The binary is compiled for x86_64.

AII-SPI-DSK project source code is now available too. The code requires user to have libusb, libftdi and libmpsse already compiled and installed. Once you get past the Xcode project format, the code should be pretty portable to other platforms. Most of the coding was done fast and dirty, so please excuse the unavoidable errors.

The USB AII_SPI_DSK Disk ][ Emulator is by far my most complex electronics project I have ever taken on. Fortunately, the Apple Disk ][ controller card is pretty flexible with the input it can take, or I would have never completed this.

I didn't quite reach my lofty goals on this project, but I did end up with a good proof of concept and a handy device in its own right. And, because this project uses a standard SPI stream, I should be able to easily create a micro-controller based solution that actually uses an SD card! The Retro icing on the cake for this project is using 2 ICs recycled from an old Apple II+ for my build.

Apple IIe Platinum: Living a new dust-free life.

Apple IIGS: Still rusty and in pieces. A little closer to living though.

Apple //c: Still neglected with 255 ROMS. I didn't even get to think about this project.

Take a minute to check out the RetroChallenge website and see what the other participants worked on.

Karateka! and Schematic

I can die happy. After spending most of last night and this evening making small speed refinements to get a Karateka disk image to work, with no success, I tried a different disk image file and... Kiai! I have no explanation why one works and the other doesn't, especially since they both work on Virtual ][. (UPDATE: Karateka only loads from Slot 6, Drive 1; I was trying to load from slot 7) In any event, my program can now transfer 2 disk sectors in a single USB transfer. Just about every disk image I have tried works now. If there is a disk image that doesn't work, there seems to be another copy of it out there that does. Not so different than loading up pirated games on an Apple IIe back in the 80's. I forgot how fun the original Mario Bros. game is. I definitely need to invest in a color monitor though.

I cleaned up my schematic. I think I could probably reduce the part count a little more, but these ICs are cheap and easy to find. Note that I broke out the write protect signal. With every bit transfered out representing a data bit, I don't have room to multiplex the write protect signal into the SPI stream.  You can also pull this to ground or the Phase 2 signal for write protected or write enabled, respectively.

1 SPI Bit per Disk Bit Achieved!

I figured I would try to get started on disk writing today, even though I doubt I can finish it before the end of the month. While I was working on it, I got side-tracked and re-visited my previous attempt to reduce the number of bits I have to send via SPI to represent a bit on disk. I put this aside after claiming success by reducing it from 4 to 2 SPI bits per disk bit at 500MHz. I was playing around with 1 bit per disk bit at 300Khz, and I was getting some errors, but it seemed close. At this speed, the read_pulse is slightly longer than the expected 1usec and the time between the start of read_pulses is shorter than the average 4usec. Then I tried 250KHz and everything just worked! At 250KHz, the read pulse is long at 2usec, but the time between the start of read_pulses is the correct 4usec. This only works due to my previous discovery of ANDing the SPI MOSI and the SPI clock lines to produce a shortened read_pulse. This probably won't work on all Apple IIs, and it may be too slow to handle disk writes, so I will keep this feature as an option for now. Unfortunately, it did not fix my latency issue, even though I am sending less data per sector. However, for the first time I can send 2 disk sectors per USB transfer, now that transfer size is down. This reduces the ~4000usec delay to once every 2 sectors. Maybe the libftdi mailing list has some input on this issue.

Good Enough

Finding ways to reduce the latency between sector transfers is getting tougher and tougher. The UM232H writes and reads packets with low latency, but when it goes back and forth between the two, significant delays occur.

I used some source code from MAME to load the different disk image formats. I had to make extensive edits to get it working outside of MAME though. ProDos and DOS 3.3 disks work fine. Most game disk images I have tried are working:

• Moonpatrol
• Choplifter
• Ms. PacMan
• Bouncing Kumungas!
• Jungle Hunt
• Donkey Kong
• Mario Bros.

Still no Karateka though.
I think it is 'good enough' for now. A computer faster than my PowerBook G4 might help. Here is video of my progress:



The LCD viewfinder is broken on the camera, so I missed my shot a few times.

I added a logging level option to the command arguments after I shot this video. I can set the verbosity at run time instead of needing to re-compile. I am working on my schematic today to make sure it is up to date.

I received my surface mount resistors for the IIgs from China earlier this week. I only needed one of each, but I could only buy 100 of each. Anybody need 198 surface mount resistors?

Speed Testing

I have been experimenting with disk images with mixed success.  ProDos based system utilities disks work fine. No luck with Karateka though :(

I fixed a few bugs that were causing the program to not move to the correct track at times. Also, I found out that libmpsse stalling was my fault - I commented out some code to limit transfer sizes thinking it didn't look right. That was causing the UM232H to get too much data and stall out. Finally, I have been experimenting to see if I can reduce latency beween data transfers. Loading a large program is still about 10% slower than running the same disk on Virtual ][. Clearly there is some room for improvement here.

FTDI has Mac OS X drivers for the UM232H. I may test out their driver to see if there is any speed to be found there. Also FTDI recently released a simpler / cheaper version of the UM232H named the UM232H-B.  $15 is a steal! You could make a Disk ][ emulator like mine for well under $20. Mine is still under $25.

Still awaiting resistors being shipped from China for my IIgs.

First Disk Image Read

I did it! I got a disk image to read on the Apple IIe! I had to make a few more changes to reduce the latency between sector transfers to an acceptable level for the Apple IIe:

• Disabled almost all of my logging
• Disabled excessive libmpsse purge buffer commands (purge rx buffer, purge tx buffer, then purge both buffers)
• Skipped the 74LS259 addressable latch
• Changed out a possibly bad 74LS00 ic pulled from an old Apple II+

I can use the Bag of Tricks program to assist with further speed-ups. Now, on to fixing up the mess of code I created trying to get this to work. I haven't even started on writing to a disk image yet. I will get a video up ASAP, but I dropped and broke my camera getting this screenshot.

Fun with Logic

While I was working on testing out different possible SPI speed settings and byte configurations, I started thinking more about ways to get my 1us read pulse while running at slower than a 1Mhz transfer speed. Then it hit me: by ANDing my data out (MOSI) to the SPI clock, I can get a read pulse half the duration of SPI data output!  At 500MHz, I have a perfect read pulse of 1us and 4us between the start of consecutive pulses!

(SCLK & MOSI) = perfect READ_PULSE

After doing some research, I realized I don't even need an extra chip to do this.  I can use a 74LS00 NAND gate for several bits of logic.  By connecting the two inputs of the NAND gate, I get the NOT gate I already needed.  To get an AND gate, I connect the SCLK and SPI output to each side of the NAND gate, then I connect the output back through a NAND gate with the two inputs connected.  NAND->NOT=AND.  It's not working yet, but I think it's close.

Shot in the Dark

So far, I have been shooting in the dark trying to get the Disk ][ emulator to work. I have no way to tell what the Apple IIe is getting as data. Here is some AppleSoft BASIC code I wrote last night (using a Beneath Apple DOS machine language program as a guide) to see what the bytes the controller card is getting. BASIC doesn't appear to be fast enough to get more than a few bytes per 'rotation' of the disk, but at least I can get a general idea of whether I am sending any valid data over. I am using this to experiment with different SPI clock speeds and byte formats.

10 SLOT6 = 49376: REM $C0E0 20 DRIVE1 = PEEK (SLOT6 + 10) 25 REM DRIVE2 = PEEK(SLOT6+11) 30 REED = PEEK (SLOT6 + 14) 40 DRIVESTART = PEEK (SLOT6 + 9) 50 DIM BYTES(255) 52 FOR X = 0 TO 254 54 BYTES(X) = PEEK (SLOT6 + 12) 56 REM IF BYTES(X) < 128 THEN X = X - 1 58 NEXT X 60 WPROTECT = PEEK (SLOT6 + 8) 90 HOME 100 FOR X = 0 TO 254 130 IF BYTES(X) = 255 THEN INVERSE 140 PRINT BYTES(X); SPC( 2) 150 NORMAL 160 NEXT X 170 END

Hardware Changes and Issues

Today I took some time to:

• reconfigure my hardware to use the 74LS161 counter
• replace the 74LS151 with a 74LS251 multiplexer (same but with 3-state output)
• switch some wires around to account for the Disk ][ sending data MSB first
• make a 2x10 (20 pin) header breakout board so I can access all of the lines from the Unidisk cable

A normal solderless breadboard has a gap in the middle that is too wide for a dual row pin header. The breakout board spans the gap.

While making all these modifications I figured out what change I had previously made to get the input from the disk controller working consistently. I was getting strange output because the multiplexer did not have time to set up the data before being read. Through trial and error, I had inverted the clock signal to the counter. Inverting the clock signal gave a half a cycle difference between the multiplexer selecting the line and the UM232H capturing the data, more than enough time for the data to get set up properly. Once I reduced the transfer size, input started working perfectly. I have now implemented this fix in software by selecting SPI mode 2 which captures data on falling clock edge. Exciting stuff, I know!

However, that wasn't the only problem I temporarily bypassed. Reducing the size of my transfers previously fixed the libmpsse stalling problem. Once I wrote functions to transfer full disk sectors, the stalling problem reared its head again. The transfer buffer of the UM232H is 1kB. A raw Apple II disk sector is 416 bytes. When I convert the sector into my SPI stream it quadruples to 1664 bytes. This is too big for a single transfer to the UM232H. libmpsse is supposed to automatically break this up into several USB transfers. Instead, it is stalling the UM232H for some reason. Even if it was working right, the buffer size is a serious problem. I can't take a break in the middle of transferring a disk sector to get more USB data - the Apple II would certainly indicate a disk error. Somehow, I need to reduce the number of SPI bits I transfer to represent a disk bit. I think I may be able to do 2 SPI bits per disk bit and slow the transfer down a little bit. This will cause the read pulse to be slightly longer than ideal and the time between pulses to be slightly shorter than ideal, but it might still work. Incidentally, this wouldn't be a problem on a microcontroller which would not have to wait for a USB transfer to get more data.

Coding all Week

This past week I spent most of my free time after work writing code. Mostly:

• a main loop
• lots of logging functions
• a function to translate the active phase line to an absolute track location on the disk image
• a function to load a NIC disk image file from disk and break it into tracks
• a function to convert a bit of disk data into four bits of SPI data that hopefully conforms to the Disk ][ read pulse

I am trying to use the NIC disk image format from the SDISKII project for now. It is the most similar to the raw stream coming from the Disk ][ drive. A normal .nib image is not much different and should be easy to implement once I get this thing working - which it certainly isn't...

If you are keeping score, my goal was to have a disk emulator that did not use a special image format, did not need a connection to another computer, and could use SD cards. At this point, the disk emulator doesn't work, it uses a special image format, it requires a USB connection to another computer, and it does not use SD cards. UGH. At least I still have a few weeks. No time to fix it this weekend, family stuff to do.

Disk ][ Controller Card Output Update!

Finally, I am able to read some consistent bytes from the controller card that look like I would expect! However, I am not totally sure why it is suddenly working...

IIGS Upgrade Change of Plans

I have been struggling to get libftdispi to output anything of use and dealing with libmpsse constantly freezing the UM232H, so I decided to try a change of challenges for the day:

I have been checking ebay for a good deal on a IIGS logic board. I finally found one for a good price and it's local so I can avoid shipping costs. However, after taking a close look at the pictures of the board, I realized that the standard IIGS logic board does not have the connectors for the Apple IIe power supply and keyboard (it does have the solder points though). Fortunately, the connectors are pretty standard. I figured though, if I am going to have to pay over $40 to solder connectors on a new logic board, I may as well try to repair my own board. So the logic board went into the dishwasher to get cleaned up. I have had good results cleaning a board this way, just make sure to dry it off completely! The board is a mess. There are vias that are corroded completely through, leaving a cumbly substance behind. There are traces that are completely gone. Some traces are corroded under the solder-mask, so you can't even tell there is a problem at first. There are surface mount resistors that have a pad corroded through. A couple of hours of mask scraping, continuity testing and soldering tiny wires got me here:

Repairs to Battery Area	Repairs to Back of Board Resistors R193 and R14 had to be removed

Now I have to find the appropriate resistors to replace the corroded ones. I can't find any surface mount components locally, so off to ebay...

Initial Schematic and Hardware

Here is the schematic that I have been working on for the past week, and my first attempt to breadboard the input portion so I can read the head control ("phase") lines. I already noticed some problems. The FTDI UM232H is master SPI only, and I drew up the schematic as a master also. I already fixed it on the breadboard. Also, I eventually need some 3-state outputs on the disk and SPI bus. Finally, I have not settled on the 74LS192 counter. I chose it because I thought I could get away without a NOT gate, but I realized I needed one anyway (the 74LS04N). A 74LS161 counter is probably more readily available (4 of them in an Apple II even), and can do everything I need. Since my last post I discovered that there are not one, but two enable lines in the DB-19 connector, so this has been accounted for. The cable is from an old 5.25" Unidisk. DB-19 connectors are very hard to find.

As for the software, I have compiled and am experimenting with libftdispi and libmpsse (which appears more complete). Unfortunately, I am not having much luck with either of them past identifying and initializing the UM232H. Both of these require libftdi which requires libusb. Using these libraries should make the program very easy or trivial to port to other platforms.

Initial Schematic	Initial Assembly - Left to right: UM232H, 74LS259, 74LS192, 74LS04, 74LS15, Unidisk cable

Apple Disk ][ Emulator Background Ramblings

While I am working on the layout of my Apple Disk ][ emulator, I thought I would give a bit of background on the Disk ][ and my plan to conquer it.

Functional Digram of Disk II

from 'Understanding the Apple II' by Jim Sather

The Apple Disk ][ was a major achievement for Apple and Steve Wozniak. They were able to replace a large number of logic ICs on a standard disk controller with a ROM-based state machine or "Logic State Sequencer ROM". The disk head is positioned by direct software control of the head stepper, rather than an absolute positioning system. To get to absolute track zero, the controller slams the head against the stopper repeatedly, causing the loud clatter recognizable as the Apple II start-up sound. This head control accounts for 4 wires connecting the drive and the controller card. The drive is enabled by an additional line that activates the drive motor and serves as a drive select. Writing is handled by two lines; the write request, which puts the magnetic head in the write state, and the actual write data serial line. There are +5v, +12v and GND power lines. Finally, there is a write protect line that indicates whether there is a notch in the disk, and a serial read pulse line. The read pulse is controlled by a special IC that outputs a 1us pulse for every field reversal on the disk. The interval between the start of consecutive write pulses is about 4us. The data transfer is asynchronous, with no clock line, but allows for some variability in when the read pulse can occur.

With this information, my plan is to convert all of this into an SPI serial stream. The 4 head stepper lines, 2 write lines and 1 enable line will be multiplexed into an 8 bit/byte serial stream (with one empty 'reserved' bit). The read pulse will be 'emulated' by 4 bits of data: 1 high bit, and 3 low bits for every read pulse. At a 1MHz transfer speed, we end up with a 1us high pulse, and 3us low, for a total of 4us. Just what the Disk ][ controller card needs! To add the write protect, I can add another line, or multiplex it into the read pulse since we have 2-3 bits of empty data that will always be low. The clock line can be from the drive controller card or the micro-controller. The main problem it that the emulator will need to transfer 4 bits of SPI data for every 1 bit of Disk ][ data. Still, this is better than my original plan of 8 bits SPI/1 bit of Disk ][ at 2MHz. The benefit is that the SPI transfer can be easily handled natively by a typical micro-controller. No processor intensive bit-banging is required. Also, we don't need a micro-controller at all to handle the timing: it is built into the SPI transfer. We'll see if these advantages make up for the extra bit shifting required.

By the way, I have decided to use the FTDI UM232H for my proof-of-concept rather than the Arduino/Atmega AVR. The FTDI chip puts out a nice square SPI wave, with no delay between bytes like the megaAVR. This is important for our timing sensitive abuse of SPI. However, this might not be an issue at the slow speed we are going. Also, I am familiar with libusb which I can use to control the FTDI chip and not so familiar with the Arduino IDE (although it looks easy enough).

First IIe Challenge Completed!

] PRINT "HAPPY NEW YEAR!"
For the first day of the Retrochallenge, I tackled what appeared to be the easiest project: getting my latest purchase, an Apple IIe Platinum, up and running. With the help of my air compressor and a clean paint brush, I managed to get all of the dust out of it. I booted 'er up and was greeted with Apple //e at the top of the screen! However, Control-Reset was not giving me an Applesoft BASIC prompt. I pulled the floppy disk controller card, rebooted and got a prompt, but I still couldn't reset the computer. I narrowed the problem down to a bad Reset key which looked bent out of whack to begin with. Some continuity testing showed that the key was not making contact. So: disassemble computer, de-solder, disassemble key, straighten the bent contacts until the key works, re-assemble key, solder it in, re-assemble computer. I replaced about five missing screws on the keyboard too. It looks like a previous owner tried to fix it and left the screws out. This was putting stress on the solder joints of the keys. I also managed to dig up a platinum Apple Mouse to match the computer and connect to the Mouse Interface card. Anyway, she works! Success on the first day!? Maybe I should stop while I am ahead...

Key removed - note missing screw for brass standoff	Key soldered in with new screws
No more dust- much better!	Target Environment: Apple IIe, solderless breadboard, Disk Interface card, FTDI UM232H or Arduino Nano?, 74LS series ICs