"It is the same thing for observer 1 to rapidly travel into the past as it is for observer 2 to rapidly travel into the future."

No, it is not. Going back in time would imply that the time order of the meetings as seen by observer 2 is the reverse of the meetings order as seen by observer 1. As presently "understood", if meeting 2 is later than meeting 1 for one observer then it will also be later for the other observer.