My DH is not a Disney fan. He doesn't like rides and just not generally into it. He went on family trips when the kids were little and as time went on discovered things he DOES like - like the food festivals at Epcot. He enjoys eating so we are careful to include interesting meals, relaxing mornings getting coffee, etc. Now that the kids are grown, we all still continue to go together but I go by myself sometimes. Our younger son is currently a CM at WDW so I can visit him and I currently need surgery to replace both knees so it's one of the few places I can vacation and use an ECV and all the lines are mainstreamed, etc.
I would be very frustrated to vacation with anyone who constantly beat me over the head with how dumb Disney is. I've also found I enjoy short, more focused trips - like 3 nights - which can work to get your Disney fix in and see the new stuff - and those are easier to do just yourself or with an older kid or friend (or the spouse who doesn't love it like you do).