When he and his brother, Richard, were young, they both tried to sign up to the navy, but were turned down because they were both too drunk and young. Later, they both fell in love with a woman named Molly, which caused their rift. For years, they grew apart because of their love for Molly. In the episode "Deadly Misunderstanding" at the very end, Jessica mentions Seth is waiting for his daughter to have a baby.
Season Eleven's "Scent of Murder" and Season Twelve's "What You Don't Know Can Kill You" seem to contradict the statement that Seth and Richard are each other's "only family," but, obviously, Richard has children, so the statement seems to be referring to surviving parents and siblings, not extended family or progeny. How Amy Walters is his niece is not clearly stated in her one appearance, but she could be his niece on Ruth's side.
During Jessica's years as a writer and mystery solver, Seth was a local doctor in Cabot Cove. He often assists Jessica and Amos Tupper with murder cases whenever they turned up. He either provided medical or intellectual information about murder victims or the murderers themselves.
In one episode, Seth participated in his late brother's murder and saw it through.