Hhm, curious

Well, the little square does more than just define the baseline....and it may be that it just coincidentally defines the baseline, I'm thinking just because of your example where it doesn't. It's also the text handle, which can be used to snap....and maybe other for other things, I'm not sure.
I know that if you change how the text is justified (left, right, centered) the little box moves (to the left, right, or center). I'm just playing around with some sample text, while I try to find some answers for you. And I see that if I rotate the text, the little box...handle, the handle rotates right along with it. Oh, ok! If you change it to vertical text (just below the justification buttons), the handle is moved away from the text, like in the text at the bottom of your example. But I can't figure out how the handle gets to be on the side, like in the text on the left side of your example.
Ok, well I've managed to get the handle on the side of the text, but I doubt this is how the original creator did it. Starting with a single digit, set to centered just., using the text tool control bar, rotate 90 degrees, set to vertical text, then using Selector tool, rotate back. But adding a 2nd digit gets trickier, because when you rotate, the spacing is all off. So clearly, this is not how it was done.
My first thought, when I read your message the first time, was that the placement of the handle must have happened in whatever program or technique the creator used. Do you have that info, or know anything about that?
And you said that when you resize and export is where this causes a problem. But I'm not aware of the text handle affecting resizing or exporting. Can you show us the result of resizing, and how the text handle is causing it?