A Raspi Zero is out of question for me.
Also its not buyable in numbers, so it can`t be used for a easily distributable product.
Maybe the existing base for tiny2313 could be used and a atmega32u5 or something.
Hey
atmega32u4 doesn't have support for usb host. Only device.
I think a combination of Arduino nano and max3421 might work? (maybe even pico running the Arduino core) Maybe it's time to find a usb host shield on amazon or something.
-Lime.