I mean, we know James is an unreliable narrator and all, and (big spoiler)
PRIME_BBCODE_SPOILER_SHOW PRIME_BBCODE_SPOILER:
says Mary died three years ago, when he instead just murdered her days before
I mean, assuming that James and Mary had been married a few years before her death, and that they went to Silent Hill early in thier relationship (perhaps they met each other there or had thier honeymoon there....)
So then, was Silent Hill a lakeside resort town before Alessa's "death"? When did James and Mary visit it?
When Mary says "You promised you'd take me there again....But you never did." , I assume that they went several years without visiting the town ('cause it's wierd to take a vacation and return a month later :/ )
And a bit of baseless speculation - James' jacket is a military green. He also wears a pin of an American flag, which also might represent some kind of military past. Vietnam, perhaps? (That might be why he's a nutcase xD)