Using generative neural nets in keras to create ‘on-the-fly’ dialogue

How hard would it be to approximate the time/length of each word in the data set? I think you could make some quite interesting videos by matching the generated monologues to the original audio clips :)