Well, personally i believe when THE ONE comes along, you will know it, and you will never let him go. Since he didn't give you any feelings, he definitely is not the one you are looking for, so DON'T let the whole "should i turn back?" idea bother you.
The real tragical thing is that the one for you does not love you. The lose and seperation because of some uncontrollable reasons could be deconstructive, if you read the poster upstairs, you will get the idea. However, i rather be that man, because by such at least i have found the one, for me.