Toward Inference-Optimal Mixture-of-Expert Large Language Models