Both of my grandfathers died before I was born. My paternal grandmother died when I was 4 and I don't remember her very well. I do remember my maternal grandmother (she died when I was in middle school) but she was not a very nice woman. I referred to them all as Grandma or Grandpa [LastName]. (I used her last name when addressing her directly.)
My kids call both of their sets of grandparents (all living) Grandma and Grandpa. Last names are only used for clarification and not generally used when addressing them directly. When my son -- the oldest grandchild -- was young, he called my father 'Pa (as a shortening of Grandpa, I guess.), my oldest niece started doing that too and for a while we thought it would stick. But they switched back to Grandpa when they were old enough to say it well. Grandpa was my dad's picked-before-the-kids-were-born name of choice, but he said he was a little sad when 'Pa went away.