The Shape of Space and Relativity

I like to read physics books for laymen. I would like to be able to read the ones for scientists, but hey, come on. How much additional math would I have to learn and how long would that take? No matter. I have trouble understanding the layman books. I would like to have a better understanding of relativity theory and quantum mechanics. However, from what I understand, even theoretical physicists have trouble understanding these. They may have a deep understanding of the mathematics involved, but have difficulty comprehending the actual physical workings of the Universe that those mathematics describe.

Even the relatively simple Special Theory of Relativity (about which Einstein wrote in 1905) is problematic for me mentally. Actually, I do understand the concept that if two people are moving at a constant rate of speed relative to each other, it is impossible for either person to know which is moving. But, hey, when I am told that for person A, the clock hanging around the neck of person B will appear to be running slower than his own clock, but for person B, the clock hanging around person A's neck will appear to run slower, my brain begins to tangle up like spaghetti in boiling water.

I recently began re-reading Brian Greene's book "The Elegant Universe". When I got to the part about the mutually lethargic clocks, I realized I could not comprehend this like I thought I did the first time around. The question always arises as to what happens when the two meet up. Who will be younger? I mean they can't both be younger simultaneously, right? The answer is that the only way for them to meet up is for one of them to accelerate by turning around and chasing after the other. That acceleration is what makes time run slower for that person in reality and thus cause that person to be younger than their counterpart. Okay, but what happens if the two meet up again without either one of them accelerating. This may sound like lunacy, but if the Universe is a closed system like a hypersphere, this is possible. Think about the Earth, If you start moving in any direction and another person starts moving in the opposite direction at the same speed, you will meet back up again after about 12,500 miles of traveling even though neither of you ever had to turn around and chase the other person down. Some scientists conjecture that space is curved in a similar way. This would obviate the need to wonder about the wall at the end of space and what's on the other side. There would be no end to space, even though its volume is finite. If you travel long enough in one direction, you will end up where you started.

But isn't this form of curved space incompatible with Special Relativity since we would have a paradox when the two space travelers meet up for the second time. Neither one of them would have accelerated, so the other person's clock, from either perspective, was always running slower than their own. Wouldn't this mean that, from both perspectives, when they meet up the other person will appear to be younger and their clocks will be be lagging their own? This just doesn't make sense.

So, am I right that either closed space does not exist or there is something wrong with Special Relativity? Or could it be that some sort of acceleration is necessary for an object to navigate the curvature of space?