"I remind myself I was lucky to have had any time with him at all."

or with her... or them...