We enjoyed both restaurants with my 5 year old niece and 3 year old nephew last summer.
DH and I have been to Sci Fi many times before and its one of our favorites. The atmosphere is unique and something you can only do at Disney. The food is relatively basic but good, ideal for lunch IMO. My niece and nephew barely noticed the movie clips. Weren't phased by them at all.
Hollywood & Vine we did for breakfast and were surprised at the food offerings and how good it was, after hearing many bad reviews. It was a pre-park opening ADR and we were one of the first up at the buffet line, so that may have attributed to the hot and fresh food. So I can't speak to lunch. The character interaction during this time was also good and more intimate, with few other tables around and the restaurant was not at capacity so a few came by more than once.
You probably can't go wrong with either but given how many other character meals you're doing, I'd lean towards Sci Fi for a more relaxed meal.