ElWhip is on GitHub

I uploaded my source code for ElWhip onto GitHub. Unfortunately, there are still some files that have Macintosh line endings, so some of the code can't really be viewed on GitHub unless you hit the "Raw" button. Hopefully it will inspire somebody to make a web browser for the original Macintosh. It will also make updates and additions to the code easier for me. Now that it is uploaded, it should make adding a log/status window quite a bit easier.

Apple //e Webserver Running!

I have my Apple //e BASIC web server running!... sort of. Try it out. Be gentle and patient, it's only 300 BAUD!: http://162.232.186.125:8899

Code after the break.. Update: minor change in code to allow OPTIONS method

RetroChallenge 2015/07 Agenda

July RetroChallenge requires moderation for me, so I will keep my challenges simple:

• Retr0bright my recently revived (and upgraded) Macintosh 128K
• Add a few features to my ElWhip web server
• Mess around with an RS232 WIFI device on an old Macintosh
  

ElWhip 0.07 with New Features

I added some features to the ElWhip web server, some of which are built in to lwip, and some are completely new:

• Dynamic File Read: The server will read the file in chunks which saves memory and allows much larger files to be in downloaded.
• Dynamic headers: The server generates the headers now and they are not required in the HTML files. This makes it easier to add images and file downloads.
• HEAD HTTP method support: The server responds appropriately to HEAD method requests.
• 'Gentle Quit' in 128K: The application will report an error rather than crashing when trying to quit with 128K RAM.

See the ElWhip Download Page for more details.

RC 2015/01 Epilogue

A member of 68kmla.org was nice enough to test out ElWhip on a real Macintosh 128K: ...and a 512K.

First time getting a REAL Macintosh 128K on the internet. So great!

ElWhip 0.06 Download and Video Finale

ElWhip Webserver 0.06 Download now available.

RetroChallenge 2015/01 Summary:

• Repaired video on my Macintosh
• Compiled and modified lwIP Web server to run on Macintosh
• Wireless logging
• Modified Mini vMac to run virtual serial ports on PPC Mac OS X
• Lubricated all of my old Mac disk drives
• First web server to run on Macintosh 512K
• First PPP connection, TCP/IP stack, and web server to run on a (virtual) Macintosh 128K

So, a ton of work, but a success, even though it could still use a little work to clean up a few things. Thirty-one years after its introduction, the original Macintosh is finally on the web! This was a great experience and I was able to learn a little too much about the Macintosh Toolbox. This was a bigger project than I expected, and it took a lot of tweaking to get everything running in 128K. Thanks for the motivation RetroChallenge!

Loading HTML files in 128K

I spent most of yesterday trying to track down a nasty bug in the lwIP http server. Most of lwIP is very well coded, and other than the CRLF problem I had, very portable and bug-free. The lwIP http server is a bit of a hodgepodge of code though. Lucky me. Anyway, the callback to release the html file resources happens before the server has sent the TCP packet. As a result, I was free()ing memory before the server was done with it which caused corruption in the HTTP headers. I decided to just keep a file buffer up all of the time instead of malloc()ing and free()ing memory as needed. This was a constraint on the size of file that can be loaded though.

I spent the rest of the night moving more PPP code into a separate code segment that I can unload once the connection is up. It worked so well, I was shocked when I saw 9K of free RAM. This easily left me 4K of RAM to load HTML files. I can even open the menus without crashing. Unfortunately, it crashes on Quit on a 128K because it needs too much RAM to bring down the interface.

So, I will get an upload ready and get a video done in the morning!

So close...

So I was able to finish the code to load the HTML files from disk using the original Macintosh File Manager calls. This saved quite a bit of RAM, and allowed be to get the server up. However, now it can't allocate enough memory to actually load a file. At least it logs the error gracefully, rather than crashing: PPP Phase Changed to: 8 App Limit: 91260 Free Memory: 9840 Starting Apps. App Limit: 91260 Free Memory: 2768 Error: "Unable to allocate memory for file." at line 105 in :::ports:mac:include:fsdata_custom.c

I am going to keep working on this today. Tomorrow is the Super Bowl, but I might have time to make a quick video, if I get it working.

Fancy Error Page

I worked much of the weekend trying to get the lwIP web server to load files from a disk drive. This could potentially save some memory because the files could be loaded on the fly, instead of all being loaded into memory at once. I had to enable a lot of additional code to get it working, which more than ate up the memory I hoped to save. Anyway, I reverted all of my code changes, and just fancied up the 404 error page that is compiled into the application. So, my Mini vMac virtual 128K is now running a little web server! I'll try to keep it up through the end of the month.

Macintosh 128K 404 Error Server!

I have been spending a few hours after work slowly trying to conserve memory here and there by delineating parts of the code so that the open and close portions of PPP are in a separate code segment. My thought was I could unload that segment once PPP is up and then load in the HTTP server into the freed memory. This is not working as planned. However, I tried removing most of the HTML pages and the image imbedded in the HTTP server, to see if I was even close on memory usage. I was able to get it serving just a 404 Error HTML Page running in a virtual Mini vMac 128K Macintosh, with just 1K of memory to spare:

404 Error page from virtual Mac 128K. Note Free Memory!
 

Even opening a menu causes a crash.

Memory is so tight that even opening a menu causes it to crash with a 'Memory Full' error code. However, this gives me hope for 2 things:

• If I load the HTML files from disk, as needed, I could probably save some memory.
• Segmenting out some more functions could give me enough free RAM to really get this running.

I will leave it running overnight if people want to try it: virtual Mac 128k 404 Error server. I still need to do some hardware modifications to get my original Macintosh back to 128K RAM. I hope I am getting close...

ElWhip 0.05 Download

I had a major disk drive failure on my 512k. My external 800k Drive doesn't spin anymore, and my other 800k drives aren't compatible with 64k ROMs / HFS. That set my testing back a bit, as I went through and cleaned and re-lubricated all of my old disk drives.

Anyway, ElWhip 0.05 is available for download. Instructions are included.

ElWhip Icon

The name of my web server application, "ElWhip" (from 'lwIP'), inspired a Zorro-esqe icon:

'El Whip' would totally be my name if were a Mexican vigilante (or luchador).

It took me a while to remember that I need to set the Bundle Bit ("Has BNDL") in ResEdit for the older versions of the Finder to correctly show the icon.

Web server ran all day today!

Mac 512K on the Internet

I fixed a couple bugs this evening, including the netmask problem I was having. I just needed to set the ppp interface as the default interface ( netif_set_default()) and lwip routes all the packets through the ppp interface, even if its router/ppp server is outside the netmask network.

I enabled ip forwarding on Mac OS X to give public access to my Mac: sudo sysctl -w net.inet.ip.forwarding=1, and set up port forwarding on my router.

And so, here it is, on the internet for the first time: my Macintosh 512K Web Server Running System 2.0 and Finder 1.1g (http://72.220.233.84:8080). It is just using the default webpage right now - I need to set up a better one. It seems to work from my iPhone. It is 30 years behind on security updates, so please be kind to the old thing!

Leave a comment if it works for you. I'll try to get a video up this weekend with more details.

Debugging Payoff

My new debugging setup, along with increasing the logging levels in lwip, quickly revealed some of the problems I am having. I managed to tackle a few of them last night.

First, I noticed my lwip ppp interface is not attempting to re-negotiate a connection. So, if it fails the first time, you have to quit the program and start again. This isn't very user-friendly, so I added a timed function to check the ppp status and, if necessary, alert the user and restart the ppp negotiation. This makes recovery from a timing problem much easier.

Next, during a failed connection to the webserver, I would get a warning like: pppos_input[0]: Dropping bad fcs 0xc540 proto=0x21
Basically, lwip was dropping ppp packets. The bad fcs was misleading and caused me some headaches. Finally, I noticed that every time this happened, about 4 lines before the packet was dropped would be a log entry: pppos_input[0]: got 63 bytes Now, lwip ppp reports got xx bytes whenever it gets data from the serial port. It is the 63 bytes that is significant, and was associated with the dropped packet. I happened to retain some information from my book, Inside Macintosh, Vol. II:

Each input driver's buffer can initially hold up to 64 characters, but your application can specify a larger buffer if necessary.

Ah! So, lwip was getting the opening part of a packet, which was cutoff by the buffer overrun, then getting another packet, ending with the checksum. The checksum wouldn't add up, so it dumped the packet (actually two packets). I think it reports 63 bytes rather than 64 bytes because the buffer is a pascal string with the first byte representing the string length(?). Fortunately, Inside Macintosh, Vol. II also provided a solution:

SerSetBuf allows you to specify a new input buffer, replacing the driver's 64-character default buffer.

So, I added a 256 byte buffer and the connection is much more reliable. It even significantly improved the rate of initial ppp negotiation.

Uptime On Old Time Macs

Too many chores to do today, so I just let the web server run on the Macintosh SE, checking to see how stable it is. Running ping -i 60 192.168.11.130 | cat -n from the Mac OS X terminal pings every minute and lets me know how many minutes the TCP/IP stack stays up. It started losing pings around 65 minutes and eventually timed out altogether. This seemed to correspond with the After Dark screensaver on the Mac SE. I turned off the screen saver (and turned down the brightness knob), and the server ran for 174 minutes before I turned it off. There were with no dropped pings and it was still serving the web page when I stopped it.

After that, I ran the web server on the original Macintosh (still with 512K RAM), and had similar success. As far as I know, this is the first Macintosh web server with 64K ROMs or 512K RAM, ever! Certainly the first on System 2.0 and Finder 1.1g.

Web Serving! - barely

I can serve a web page from the Macintosh SE! There was a problem where lwIP does not seem to be detecting the first CRLF (Character Return + Line Feed) required by the HTTP spec. So, when I used Telnet, I had to hit return 3 times before the HTML appeared. Web browsers will only send 2 CRLFs, then wait. I ended up deleting one of the checks for a CRLF from the code, and it worked! However, that doesn't explain why it isn't detecting the first CRLF. I think it could losing the last character of the buffer (LF) or something.

The process of dealing with this made me realize I can't keep overlooking a few problems I have seen:

• Getting PPP to connect is too timing-sensitive and flakey
• I can't always get the web browser to connect
• PPP is a memory hog
• I need to actually fix all of the "FIX ME"s I have been inserting into the code
• I need a better logging / debugging system

Some of these problems are related. I need a good debugging system system before I can fix any of them. Enabling debugging currently causes a bunch of modal dialog boxes to appear. This causes further timing problems and does not allow logging for analysis.

Once I get some of these issues sorted out, I will test it out in 128K and make a download available.

Macintosh Webserver Recap and Restart

lwIP is a lightweight TCP/IP stack that was designed for embedded systems with, or without an Operating System. During RetroChallenge 2014WW, I was able to get MPW 3.5 compiling programs for old Macintosh System versions, port lwIP, and get a TCP/IP stack running on my Macintosh with 512K RAM. For RetroChallenge 2014SC, I was able to update my working version of lwIP and compile the webserver before life interfered.

---

I woke up this morning without much of a hangover, allowing for an early start on the challenge.

I updated my local copy of the lwIP source which has now incorporated the ppp-new branch into the main branch. It is nice to see continued development on lwIP. This updated ppp code was important because it allowed for ppp without multithreading, which is necessary for the early versions of Macintosh OS. With git installed, two quick commands did the trick: /usr/local/bin/git clone git://git.savannah.nongnu.org/lwip/lwip-contrib.git /usr/local/bin/git clone git://git.sv.gnu.org/lwip.git

After copying over my additions, and converting the end of line characters to be Mac friendly, I was able to compile it with MPW again.

I fixed the error about the fsdata source file not needed for link. As it turns out, that file is included through one of the header files, and does not need to be compiled separately as a source file.

Finally, I transferred the program to my Macintosh SE for testing (thanks ZIP drive). The PPP and TCP/IP connections come up fairly reliably, but, I couldn't connect to the webpage using Safari.

I Telnetted to port 80:
PowerMac-G4:/Volumes/Mac OS 9/Maclwip151 epooch$ telnet 192.168.11.130 80 Trying 192.168.11.130... Connected to 192.168.11.130. Escape character is '^]'. GET / HTTP/1.0 404 File not found Server: lwIP/1.3.1 (http://savannah.nongnu.org/projects/lwip) Content-type: text/html <html> <head><title>lwIP - A Lightweight TCP/IP Stack</title></head> [...]
And it seems to have worked! I will have to keep testing this evening and look through the code a bit to see why it is not loading with Safari (or FireFox).

I am already further along than RetroChallenge 2014SC!

Retrochallenge 2015/01 Entry

Retrochallenge is back!
I seem to be able to get more done during the Winter Warm up than the Summer Challenge, so I am feeling ambitious. I still want to get a web server working on my Mac 128K, so that is my primary goal.  I have renewed my interest in working on my 1991 Nissan, and interfacing with the ECU using the Nissan CONSULT protocol.  It is a JECS microprocessor which is either a Motorola 6802 or Hitachi 6303 equivalent. It isn't the typical retro computer, but it seems to qualify.

Slow Start

Between 4th of July, World Cup, and the British Grand Prix, I haven't spent much time on my RetroChallenge project. I spent way more time reading about other people's projects than working on my own! I did manage to compile the latest lwip sources and the lwip httpserver into an application, but I haven't tested it yet. The MPW build option to use unix #include paths worked perfectly.  I wish I had know about it last time around, as it would have saved me some time.
SetDirectory "{MPW}Maclwip2:lwip-contrib-ppp-new:ports:mac:"; BuildProgram ElWhip # 2:50:02 PM ----- Build of ElWhip. # 2:50:02 PM ----- Analyzing dependencies. # 2:50:02 PM ----- Executing build commands.   SC [...]   Link -o ElWhip [...] ## Link: Warning: File was not needed for link: (Error 52) :::apps:httpserver_raw:fsdata.c.t.o   Rez -rd -o ElWhip ElWhip.r -append   DeRez "Mac OS 9:MPW:Libraries:Libraries:SERD" -only SERD   Rez -o ElWhip -append # 2:51:51 PM ----- Done. ElWhip The warning about the file not needed for link is troubling because that is where the html for the web server is stored. I will test it all out tonight.

Making an lwip web server

Looks pretty easy:
http://lwip.wikia.com/wiki/Sample_Web_Server
although the nonstandard file system seems weird.

Both the lwip core and the ppp-new branch have been updated since January, so I will try to update them first. Also, I will try to use the MPW compiler flag for using Unix paths.  I had no idea this existed when I ported the code in January. Hopefully, this will make my implementation standard enough to include in the lwip contributions branch.

Also, I did some research on how I can test the web server with 128K of memory (rather than the 512K my computer was upgraded to).  It looks like I can just solder a jumper to disable one of the RAM address lines, making it 128K.