The Portable was a cool design and I sometimes wonder if its ROM couldn't be adapted to something like the Plus or SE, mainly because it would nice to allocate 8 MB of the 16M memory space to RAM, instead of only 4M.
I think the lack of 16MHz data bus on the SE would limit your performance. And like the Portable, if you intended to use the PDS slot, you'd be back to the stock SE configuration. You could theoretically fit a PB 100 in the SE case, but not the Portable, without which you'd lose your SE PDS (unless you were using it for a modem in which case the 100 would arguably be an improvement). It should be possible to convert the 100's a simple 1-bit digital video signal for the SE's CRT.
I'm not a big fan of the Portable's design or the Snow White stylings in general. But I do like the Portable's LCD screen and advanced 68000 features. With the screen folded down, it makes for clean desktop storage when not in use and an external monitor and keyboard/mouse make it about the same setup as an LC (and probably more powerful). Plus I love trackballs. However, I would miss the quaintness of the sing-song 400K drives which are not compatible and the early MFS Systems, but I wouldn't miss the fans introduced with the SE & Classic. As a desktop I would prefer it to be closer to the Apple //c form factor with an external mouse, but if I had to transport it a lot, I would definitely prefer it as is, rather than shoving the separate parts in a bag like the SE.
But let's compare them anyway:
_____________SE____|_PORTABLE_|_CLASSIC__|__PB 100__________
CPU_________8MHz______16MHz______8MHz_____16MHz
Bus_________8MHz______16MHz______8MHz_____16MHz
Display_____512x342____640x400____512x342___640x400
Type________CRT______Active Mtrx____CRT_____Passive Mtrx
Ext. Disp._____PDS______Dedicated____NO________NO
PDS___________1_________1_________NO________NO
ROM EXP._____NO________YES________NO________NO
RAM (MAX)____4MB_______9MB*______4MB_______8MB
MODEM_______PDS_____Dedicated_____NO______Dedicated (no Ext. Modem port)
Ethernet______PDS_______PDS________NO________NO
SOUND OUT__Mono______Stereo______Mono______Mono
BATTERY______NO_______YES_________NO_______YES
FAN__________YES_______NO________YES________NO
*5MB if PDS used (8MB & 4MB respectively on M5126), with or without backlight upgrade.
All other ports are the same.
Clearly the Portable wins hands down and the Classic & PB were indeed poor stepchildren and the SE suddenly doesn't seem so much better than the Classic. The Portable had dedicated internal connections for everything but Ethernet which could be installed in lieu of additional RAM and still have 1MB over the SE. With the SE, this choice was nonexistent. Moreover, while the SE provided for all of the functionality of the Portable via its PDS slot, it only allowed one of the options otherwise native to the Portable.
The reality is that Apple did not pursue either the external video or PDS expansion, leaving both to third parties. Therefore, external video adapters are rare and the only PDS cards available are RAM cards (which oddly Apple attempted to discourage). So the practical reality is the SE wins in external expansion options, but the Portable wins in speed and RAM. The ultimate winner depends on which is more important. But it cannot be discounted that both options are there if one is industrious enough to exploit them. The video adapter should be easy enough to make. The PDS is a little more involved and with SCSI Ethernet options, probably not worth the RAM sacrifice. However, I can't help wondering if the SE PDS cards could be adapted for use in the Portable.