Where are you located? If you're in driving distance of NY, I would recommend Cooperstown. Cooperstown is a small town but loaded with baseball history and of course home to the Baseball Hall of Fame. There's usually games you can catch at Doubleday Field. There's some great boating and water activities on Otsego Lake right in Cooperstown. Just a 90 minute to 2 hour drive away is Lake George and the Adirondack mountains with fabulous hiking and outdoor activities. Just beyond that, another 90 minute drive, is Lake Placid which has some history around with the Olympics and Miracle on Ice.
If you'd have to fly, Cooperstown would be about in the middle of the Syracuse and Albany airports. Its also a couple hour drive from Boston, which would be another good place to visit for more culture and pro baseball if its important to catch a game.
I know you said American city, but if you have passports, Toronto fits the bill as well. Toronto Blue Jays have a great stadium in the heart of the city, and the Hockey Hall of Fame is nearby. There's some outdoor activities on Lake Ontario and Niagara Falls is about a 90 minute drive. On the U.S. side of Niagara Falls, there is a state park with some outdoor activities like Cave of the Winds, and the Maid of the Mist and Whirpool Jet Boats are on both the American and Canadian sides. There's also great hiking in the gorge on the American side.