Thank you all for the replies, learned something new today
Another thing that is sort of related, there were a few things made that were semi-independent of the host. Couple of Apple-made examples :
The Apple Display 8•24 GC video card has an AM29000 RISC processor with a lot of grunt and runs a portion of the graphics software natively on the card. It is also capable of block transfers to some other dumb cards and direct accesses their framebuffers through through NuBus with no involvement of the host.
Some Apple Ethernet cards had a 68000 processor and ran their own RTOS. They were based around a system called "A/ROSE" (after a legal challenge by Microsoft made them change the name from MR.DOS... A/ROSE by any other name is still MR.DOS) which was planned to be used as sort of an SDK for developers to produce cards using A/ROSE to interface with the host and bolting their own hardware onto the far side. It wasn't a huge success because they were expensive with basically an entire computer on the card. Also, personally, I have found the A/ROSE host side software to be some of the most incompatible and remove it on sight.
Other related things, it can be hard to spot because apple generally get their chips custom labelled, but the ADB controller is a MicroChip PIC MCU, and the Egret and CUDA chips are 68HC05s in disguise.
There is lots going on there, but it feels like people mainly just surprised Macs didn't have DMA floppy control and sideways scrolling hardware.
Regarding built-in video, for general use it tended to be a fast framebuffer and not have any acceleration features (people bought expansion cards for accelerated graphics). They were generally pretty fast (especially at 8bit) in their day although even Mac users have forgotten this, in part due to all the enthusiastic marketing by 3rd parties. 3rd parties struggled to exceed the performance of Quadra video, even with accelerated routines, because of the expansion bus speed limitations for bulk transferring data, while the onboard video used 32bit word aligned dual port memory at the full bus speed for 24bit colour and heavy software and hardware optimisation. Sort of the RISC attitude - simple but fast. End result is they can throw frames to screen pretty quick, but can't compete with an Amiga WRT manipulating what is already in the framebuffer.
This is made more stark by the lack of a low resolution mode on macs - I do wonder if this was originally intentional to actively avoid being "just for games" as I know Apple feared that the GUI would be perceived as a gimmick/ toy and the Mac not a serious computer. Also the user interface was designed for approximately 72dpi, so reducing the resolution under this would have likely looked bad
I'm not sure how third party cards compare to an Amiga, I'd guess they do well in some situations and bad in others. An example of where Macs and Acorns (another platform that went for simple hardware) tend to do well and I understand Amigas struggle is "high resolution" 3D - for example flight simulators.