I'm not a good person to ask about that because both of my kids have gone to disney eveyr year since the womb! Seriously we took them both each year they've been alive! 6 months was the youngest I took a child and it wasn't too bad. Honestly it's more of a question of how much work you want to do while you're there? Diapers, bottles, stroller, whinying, sticky, naps - you have to think about that!
Would I do it again, probably not. I would probably wait until they were 4 yo. There's a reason why they're free under 3 - it's not really worth it. Trust me my kids LOVE Disney and can't wait to go back. Do they remember every trip - nope. Do they remember all the special things I did for each trip - nope. Do I consider each trip a waste of time/money - kind of/not really. Each trip was a family rejuvenation for us. When we go to Disney we're all the same age the second we pass under the train station!
We took the entire family, mil, bil, sil, my parents on a trip in December 2006 - do you think DD8 remembers it - kind of - she was 4yo. It's the memories we talk about that she remembers.
You have to do what's right for you and your family. These are memories, regardless if they remember them or not. You do and when they're older screaming at how much you make them miserable
, you'll have those memories to fall back on.