Minigpt4 Inference on CPU

I know it's not the main point of this, but... so many multimodal models now that take frozen vision encoders and language decoders and weld them together with a projection layer! I wanna grab the EVA02-CLIP-E image encoder and the Llama-2 33B model and do the same, I bet that'd be fun :D

I am not an ML expert. I want to know how to add my own documents without sending them off to a 3rd party.

Not heard of minigpt4. Why that name? Is it claiming to be specifically a gpt4 competitor?

Any data on inference speed? I’ve found that the non quantized model was much faster on GPU than the quantized versions due to lower GPU utilization.