just a thought here
anxiety cuts off ones ability to use higher level process in the prefrontal cortex, so one won't remember things so well. the cortisol that is produced when one feels anxiety also does the same sort of thing, as well as many other long term bad health effects on the rest of the body.
it is not surprising we have memory loss from this stress, then.
also, many people experience trauma in childhood but don't remember it due to the saving effect of amnesia. I am not saying everyone has this experience. just that it is common when someone has missing periods of time from childhood.