As I understand it, you should be able to run a 5300 off a 3400 battery, but it won't charge it.
Back in the day, Sony's battery division built the new high-capacity LiIon cells for the 5300, but production problems (likely stray metal shavings causing internal shorts, a problem that would recur in the early '00s with the same result: fires and a massive recall) meant they overheated and caught fire. The first-run 5300s were provided with a System extension-based hotfix on a floppy disk. Basically this extension checked at boot and updated the PMU's firmware to prevent it from charging a LiIon battery, if it ever was used (most were recalled and never made it to customers). This fix was later built into System 7 so you didn't need to use the installer anymore, but this also meant any subsequent System updates prevented use of LiIon batteries in a 5300, even when they were safe.
I think the PMU firmware update for the earliest 5300s was kept in the chip's RAM; it wasn't permanent, so the update had to check and update it on every boot. You may be able to get a very early 5300, install the very earliest version of System software without the hotfix, let it sit unpowered for a while (so the PMU drops its updated program), and try it with a 3400 battery. It may charge it. Later 5300s may have had the updated PMU firmware hard-coded into the chip so it may never work with a LiIon.
