I had a brief discussion with the guys doing the 'We've got Worm' podcast about the whole Mary Sue thing, mostly in light of the recent Star Wars movie.
There's really two sides of this - one sort of runs on my personal definition of Mary Sue. That is - the character is written in such a way that they distort the work around them. They take the setting's established expectations or rules (including the rules of, say, basic physics, genetics, science, sociology that it says yeah, sure, we're using those basic expectations) and they bend or break them. They take the established or expected characterization and bend or break them - people don't act as they should when the Sue is in the room. They take the narrative flow and, again, bend or break it. They take the system conceits (boys can't be witches, girls can't be sorcerers) and break it (he's both!), muddying the waters and making the underlying structure of the story that much less defined.
All the way down the line. You can have Sues that are sues by virtue of how they impact worldbuilding, tone, verisimilitude, prose style, meta conceits, and so on.
Taken like this, I just really feel like you can peg a lot of the Sue-ish stuff that flies under the radar and it doesn't set too high a bar to pass - the question you're really asking is... does the character have an abusive relationship to the work?
Not all sues are overpowered or super pretty/handsome. Not all sues are self-inserts, or found in fanfiction. These elements are common in Sues though, because they're unhealthy.
Expanding on that thought - you can have a Sue that's Sue-ish because of how miserably they're treated if that treatment is such that it over-emphasizes the character or if it breaks narrative & expected characterization to happen for the sake of miserable character's character (if literally nobody, even characters we expect should have some sympathy, is capable of being nice to said character), if the story is about 'everyone hates me', and is solely an excuse for the character to angst until everyone finally grinds them into the dirt, break them & they get killed soooo tragically, the end.
I said there were two parts to this. I think there's a separate question to be asked, but it does play into the Sue designation in a passive way, and it's a good thing to focus on regardless - recent posts in the thread seem to touch on it: "A heroic character is defined by their flaws, a villainous character is defined by their strengths."
It's a writing convention that exists because we know the hero is going to win. We have an awful, awful lot of experience with media where the good guy comes out ahead in the end. So the trick for the author is to paint things in a way so there's tension regardless. That gives us one good reason to establish the negative qualities of the hero in a way that matter. We know the villain is probably going to lose, and taking the saturday morning cartoon route and having negative forces be burdened with negative qualities is like painting with black on black - you're going to struggle to create a picture that's anything beyond two dimensional.
Instead of painting with black on black for the villains and white on white for the heroes, we do white on black and black on white. Shades of grey are good too. People will be using these things to figure out your characters and they'll quickly realize if you're adding a flaw that has no substance to it - if it doesn't impact the narrative or really challenge the character. If your character is really bad with kids but never deals with kids, it doesn't matter. No, you throw them into a plot with a kindergarten fight club and force them to unite the kids against the academic establishment.
A white-on-white, perks-piled-on-hero protagonist that's been afforded all of the advantages can be indicative of a Sue. White on white is glaring and readers are going to see that glare, and they're going to look for the contrast and the things that bring the end of the story into question. If they don't find those things, then it starts to bend & break rules & expectations - the natural arc of a story, their characterization, the rules & bounds of the setting, verisimilitude, etc. The things that motivate someone that's writing a hero without meaningful flaws often lead to other issues on other fronts, too.
(And I gotta say, a lot of this feeds into my personal issues of taste & lack of depth that I see in a lot of light novels, but that's a topic for another time)
To answer your question, AdamBo - when you describe your character, you struggle to assign flaws. You're protective of the overarching character. Based on what you're saying, it's very possible they could be perceived as a Sue, or at least as a flawed bit of writing.
Your character came from somewhere. They had good days and bad, and not every problem they faced in the past (whether they were three, five, ten, fifteen, or twenty years old) was 100% resolved, so they carried some stuff forward. We never have perfect coping mechanisms, we have hot buttons, and if you wrote up a character sheet for anyone there would be pluses here and minuses there. So they arrive at the start of a story with some issues & weak points. Figure out the direction you're going as you define the character and then figure out where their worst days & weak points were rooted, and work from there, carry it forward, in hints and threads and ultimately by having scenes that challenge these weak points - scenes the character doesn't come out ahead in. Hammer at the weak points, let them expand or develop, show who the character is in how they deal with the bad days & the added strains.
Often the flaws align with story in a subtle way that comes to a head at the climax, so you could start from there, figure out what the main confrontation & challenge in the story is, and then use that to go back to your character & figure out where their weaknesses are.
But if you try painting with white on a white canvas, you're going to struggle to show depth, you're going to struggle to convey a challenge/crisis that makes the reader think that hey, this time the good guys might not come out ahead, and your character may end up testing credulity & convention to the point that it comes across as abusive to the overall work.