• Updated 2023-07-12: Hello, Guest! Welcome back, and be sure to check out this follow-up post about our outage a week or so ago.

SCSI to Ethernet Adapter on New Hardware

saybur

Well-known member
The lack of auto-mounting is probably that INQUIRY thing: as I recall the current version may not be spec-compliant. I added an issue on Github to remind me to review it.

There is a buffer on the debugging USART, and as long as it doesn't saturate it should only be a ~150ns hit each symbol. I've also found the performance hit to be minimal. On the other hand, parity does slow things down: I think I found it was ~33% worse, at least the last time I checked. That part of the code could use some optimizing.
 

Chopsticks

Well-known member
hey saybur, been pretty busy with university lately, so haven't posted in a little while, just wanted to see were things are at and what I can do to help with testing etc..
 

saybur

Well-known member
Not too much to report right now: IMHO the main branch on Github is in pretty good shape at the moment. I'm working through a few side puzzles on this project when not playing an excessive amount of Hades. Most of the current progress is on the 'devel' branch on Github:
  • Built some hand-tuned AVR assembly routines for the hard drive transfers, which have improved read/write performance by ~10% in my testing. They also have support for resolving SCSI bus lockups, but that part isn't quite done yet.
  • Memory savings in several places to stretch the 64A3U SRAM budget.
  • exFAT support is now on via -DUSE_EXFAT.
  • Some other more minor items, including some code to try and detect stack corruption issues.
I've also done some unpublished work expanding the memory buffers, to squeeze out more Ethernet performance. That is very unstable and is what I'm working on today (again, this has been ongoing). Thus far no silver bullets for the TCP retransmit issues, which is too bad. This particular item may never see the light of day at the rate it's going, lol.

Another thing: my test board was stable when I overclocked the AVR to 40MHz (up from 32MHz), which is as fast as it can go without violating the SPI bus speed on the Ethernet chip. That resulted in overall improvements to performance. If I can verify that is generally stable the board will likely gain a 'overclock=yes' option in scuznet.ini. The people over at AVR Freaks seem to think these chips are pretty much solid up to 48MHz, but I'm not a fan of pushing things intentionally outside their specs without being up-front about it. :)
 

superjer2000

Well-known member
Another thing: my test board was stable when I overclocked the AVR to 40MHz (up from 32MHz), which is as fast as it can go without violating the SPI bus speed on the Ethernet chip. That resulted in overall improvements to performance. If I can verify that is generally stable the board will likely gain a 'overclock=yes' option in scuznet.ini. The people over at AVR Freaks seem to think these chips are pretty much solid up to 48MHz, but I'm not a fan of pushing things intentionally outside their specs without being up-front about it. :)

The overclocking angle is interesting!

I did some tests of the (then?) current version a week or so ago trying out the nuvolink again -

I tested on my PB 145B and got ~104kb/s down and 80kb/s up on Appleshare and about 100kbps with Fetch. I did get a freeze at shutdown. (I also had a freeze on my SE/30 at shutdown but included a bus error bomb).

I tried it on my SE (not SE/30) and only got 9kbps on AppleShare which is a fair bit slower than the 40 kbps I am getting with the current fork of Daynaport which I thought was odd.

I also tried the current Daynaport fork with my Scuznet WiFi edition on my PB 145: 90 kbps down and about the same with Fetch. However AppleShare up wouldn't really work properly (it was incredibly slow) indicating some transmission errors. I haven't investigated this yet as the only way I can think of to troubleshoot it is to use my managed switch to push all packets through to a single jack so I can see what's happening as otherwise I can't see the packets. TCP up and down seemed to work perfectly though given the speeds I was getting. I also haven't had a chance to see why I was getting similar problems when uploading via AppleShare on my computers connected to a hub (rather than a switch)- I plan to put my laptop on a port there to see what's happening as I'm thinking it might be the same issue as I'm having with packets going over WiFi.
 

aladds

Well-known member
I can't seem to make the current development branch for hardware rev 2, it gives me the following error:

Code:
phy.c: In function ‘phy_get’:
phy.c:408:34: error: expected string literal before ‘,’ token
asm_data_in_def, asm_dclk_def, asm_doe_def, asm_req_def, asm_ack_def
^
make: *** [<builtin>: phy.o] Error 1

If it set it back to V01 it compiles fine.

Otherwise I am so very impressed with all of the work you're doing @saybur - thank you for all your hard work :)
 

saybur

Well-known member
Good catch, I've only been testing on V1 boards so far. The issue is the lack of a DCLK signal on the V2 boards. I'll get the code fixed this weekend. If you want to try it out in the interim, it will compile fine if you strike "asm_dclk_def," (including the comma) in that line.
 

saybur

Well-known member
I tested on my PB 145B and got ~104kb/s down and 80kb/s up on Appleshare and about 100kbps with Fetch. I did get a freeze at shutdown. (I also had a freeze on my SE/30 at shutdown but included a bus error bomb).

I tried it on my SE (not SE/30) and only got 9kbps on AppleShare which is a fair bit slower than the 40 kbps I am getting with the current fork of Daynaport which I thought was odd.

The freezing at shutdown seems to be a persistent source of trouble. I think I recall (maybe in the vanished posts) you mentioning some changes made to your branch that addressed that, or a theory on the subject? I haven't been experiencing the issue myself but that may be down to me being inattentive.

As for the slow speed, I haven't had a chance to pull out my SE yet and redo speed tests with the current main version. I'll take a look at that and see if I can verify your results.

I have had zero luck getting the unpublished memory buffers branch working and I think I'm ready to properly drop that idea, it's been a black hole of trouble and has been highly discouraging. Hopefully implementing your Daynaport packet-stuffing discovery should be easier, which is next up on my plate.
 

saybur

Well-known member
After a bit of a break from the project, I'm back with a bootloader. This is generic enough I made a separate project for it, located here:

https://github.com/saybur/mcxboot

A version tailored for scuznet is also on the main branch under the "bootloader" folder.

This bootloader should greatly simplify the programming process. Instead of needing to use a PDI programmer each time there is a firmware update, you will be able to do the following:
  1. Do a make to create scuznet.bin
  2. Copy that file to a FAT32 formatted memory card (I'd suggest using a separate card from your regular one).
  3. Swap out your regular memory card for the new one with the .bin file on it.
  4. Power the device on. It will automatically begin the update process.
  5. Once the update is complete, put your regular memory card back in.
  6. Reboot the scuznet. It will boot up into the new firmware.
This has been working well for me so far. It has been nice not having to take apart my boards whenever I want to try something out, and should make the whole update process much more user-friendly.

If anyone is interested in experimenting with this on their own boards, definitely let me know how it goes.
 

aperezbios

Well-known member
After a bit of a break from the project, I'm back with a bootloader. This is generic enough I made a separate project for it, located here:

https://github.com/saybur/mcxboot

A version tailored for scuznet is also on the main branch under the "bootloader" folder.

This bootloader should greatly simplify the programming process. Instead of needing to use a PDI programmer each time there is a firmware update, you will be able to do the following:
  1. Do a make to create scuznet.bin
  2. Copy that file to a FAT32 formatted memory card (I'd suggest using a separate card from your regular one).
  3. Swap out your regular memory card for the new one with the .bin file on it.
  4. Power the device on. It will automatically begin the update process.
  5. Once the update is complete, put your regular memory card back in.
  6. Reboot the scuznet. It will boot up into the new firmware.
This has been working well for me so far. It has been nice not having to take apart my boards whenever I want to try something out, and should make the whole update process much more user-friendly.

If anyone is interested in experimenting with this on their own boards, definitely let me know how it goes.
Excellent! Well done!
 

aladds

Well-known member
I've just tried this out with some of the boards I made and it works really well - I'm very impressed!

(Anyone with a board I made but without a PDI programmer DM me and I'll program it for you for the cost of return postage)
 

saybur

Well-known member
Thanks!

More news: experimental (and I mean experimental) CD-ROM support has been added to the "devel" branch on Github, based on the SCSI2SD implementation. In the config file, define a [hddX] section like normal, but use "type=cdrom" and provide an ISO image as the image. When the system boots scuznet should present it as a valid CD-ROM.

I'm still working on this and caveats abound for the implementation:
  • The ISO used for testing so far is a backup of a real Mac install disk. I haven't tried fancier disks, nor have I tried booting off it yet.
  • Ejecting does not work yet (SCSI CMD 0x1E not implemented). Fortunately, apart from producing a type -50 error this doesn't seem to make the Mac too grumpy.
  • I'm not 100% on the configuration syntax. There are likely better ways to define things, like pointing to a folder of images with a "default" one to mount at startup, and some way to switch between them... but that's a longer-term goal and I don't know if Mac-side software exists to manage such a setup.
There are likely many bugs hiding in there, but at least it's available for brave souls to test. :) Any feedback would be welcome.

As an aside, I have not forgotten getting Dayna multi-packet support from @superjer2000 added. That feature should hopefully be next.
 

superjer2000

Well-known member
Thanks!

More news: experimental (and I mean experimental) CD-ROM support has been added to the "devel" branch on Github, based on the SCSI2SD implementation. In the config file, define a [hddX] section like normal, but use "type=cdrom" and provide an ISO image as the image. When the system boots scuznet should present it as a valid CD-ROM.

I'm still working on this and caveats abound for the implementation:
  • The ISO used for testing so far is a backup of a real Mac install disk. I haven't tried fancier disks, nor have I tried booting off it yet.
  • Ejecting does not work yet (SCSI CMD 0x1E not implemented). Fortunately, apart from producing a type -50 error this doesn't seem to make the Mac too grumpy.
  • I'm not 100% on the configuration syntax. There are likely better ways to define things, like pointing to a folder of images with a "default" one to mount at startup, and some way to switch between them... but that's a longer-term goal and I don't know if Mac-side software exists to manage such a setup.
There are likely many bugs hiding in there, but at least it's available for brave souls to test. :) Any feedback would be welcome.

As an aside, I have not forgotten getting Dayna multi-packet support from @superjer2000 added. That feature should hopefully be next.
CD-ROM support is a great add! While there may be better ways to do it, I wonder if it would be fairly straightforward for one of our Mac programming gurus (@cheesestraws, @Scott Squires , @Crutch among them) to write a Mac App that would look for a Scuznet device, issue a custom scsi command to get a list of available CD rom images and then allow an image to be selected for mounting by sending back another custom scsi command.
 

cheesestraws

Well-known member
From a cursory glance at the SCSI chapter of IM: Devices, looks like it'd be doable moderately straightforwardly... (he said, based on really very little evidence)
 

saybur

Well-known member
That would be a sweet solution. I wonder if it might be useful to the RaSCSI folks too? Some kind of Mac program that could execute commands to list/swap images ought to be useful to any CDROM emulation hardware where there is control over the command set.

I have zero experience with Mac software development, but if someone wanted to take this on I'd be thrilled to support it on scuznet.
 

aladds

Well-known member
After a bit of a break from the project, I'm back with a bootloader. This is generic enough I made a separate project for it, located here:

https://github.com/saybur/mcxboot

A version tailored for scuznet is also on the main branch under the "bootloader" folder.

This bootloader should greatly simplify the programming process. Instead of needing to use a PDI programmer each time there is a firmware update, you will be able to do the following:
  1. Do a make to create scuznet.bin
  2. Copy that file to a FAT32 formatted memory card (I'd suggest using a separate card from your regular one).
  3. Swap out your regular memory card for the new one with the .bin file on it.
  4. Power the device on. It will automatically begin the update process.
  5. Once the update is complete, put your regular memory card back in.
  6. Reboot the scuznet. It will boot up into the new firmware.
This has been working well for me so far. It has been nice not having to take apart my boards whenever I want to try something out, and should make the whole update process much more user-friendly.

If anyone is interested in experimenting with this on their own boards, definitely let me know how it goes.
Not sure if I'm doing something wrong...but I can't get the bootloader to flash a firmware more than once; it would seem that the second time I try (with a different file, in case that matters) it doesn't try to load the firmware image and instead boots as normal? I've been trying with the dev branch versus main, hardware rev 2 (192a3u)


In the config file, define a [hddX] section like normal, but use "type=cdrom" and provide an ISO image as the image. When the system boots scuznet should present it as a valid CD-ROM.
Can I get a full working example of this? I can only get it to throw an error code at the moment; this is the section I added:

Code:
[hdd2]
id=4
type=cdrom
mode=normal
file=<filename>.ISO

I've confirmed I'm using the right firmware, and the file is a valid iso. I've tried a few with the same error. The code seems to throw an error on the "id" line, but then again my counting may be off (it's around like 96 in the file!)
 
Last edited:

aladds

Well-known member
Got it!


Code:
[hdd2]
id=4
mode=cdrom
file=<filename>.ISO

Can't get it to mount anything but it's at least showing up now :)


EDIT: I tried yet another iso and this one works! I'll take a look at what the differences are between this and the others
 
Last edited:

saybur

Well-known member
Not sure if I'm doing something wrong...but I can't get the bootloader to flash a firmware more than once; it would seem that the second time I try (with a different file, in case that matters) it doesn't try to load the firmware image and instead boots as normal?
Hmm, that's not good. It makes me concerned that the bootloader is being overwritten somehow. Does it show any signs of allowing you to flash the first file again? It shouldn't care if they are identical.

I'll do some checks on my boards too and see if I can replicate this.
 

saybur

Well-known member
I experimented a bit and haven't had a situation where the bootloader overwrites itself (so far): it seems to be OK flashing images repeatedly. A couple things worth trying:
  • If you've used a PDI programmer to flash the application firmware any time after installing the bootloader, by default the scuznet Makefile runs 'make flash' with the avrdude '-e' flag. That executes a chip erase, which will trash the bootloader (not a great default). Try re-flashing the bootloader, then running the application flash off the SD card a few times.
  • To be 100% sure there isn't a bootloader self-overwrite going on, you can add '-U lock:w:0xBF:m' to the end of the 'make fuses' section of the bootloader Makefile when installing. That will prevent the bootloader from being written again until the chip is erased. I've added this to the mcxboot Github repo, it's a sane default for safety.
  • Verify that the flash drive is FAT32 formatted and that 'scuznet.bin' doesn't have any weird symbols, spaces, or whatever in it.
 

Chopsticks

Well-known member
After a bit of a break from the project, I'm back with a bootloader. This is generic enough I made a separate project for it, located here:

https://github.com/saybur/mcxboot

A version tailored for scuznet is also on the main branch under the "bootloader" folder.

This bootloader should greatly simplify the programming process. Instead of needing to use a PDI programmer each time there is a firmware update, you will be able to do the following:
  1. Do a make to create scuznet.bin
  2. Copy that file to a FAT32 formatted memory card (I'd suggest using a separate card from your regular one).
  3. Swap out your regular memory card for the new one with the .bin file on it.
  4. Power the device on. It will automatically begin the update process.
  5. Once the update is complete, put your regular memory card back in.
  6. Reboot the scuznet. It will boot up into the new firmware.
This has been working well for me so far. It has been nice not having to take apart my boards whenever I want to try something out, and should make the whole update process much more user-friendly.

If anyone is interested in experimenting with this on their own boards, definitely let me know how it goes.
hi saybur,
I have been absent for quite awhile due to a busy workload with school, just wanted to say that I think its great you're still working on this and have a bootloader work, also I noticed a few post later that you have some initial CD-ROM support working.
I have a couple weeks free from study atm so please let me know if you need any testing to be done on an SE/30 as I'll be happy to do so
 
Top