Yeah, I'm not assertive on a regular basis. But if I get pushed enough, you can get me pissed enough to actually stand up for myself! But as for your sister, she sounds a lot like me. I'm almost nice to a fault... but it's really out of fear, not because I want to phase myself out or get walked on (figuratively or literally!) or anything. So I guess I can relate.
In an odd way, I have to thank my family for nitpicking and doing little shit here and there to set me off. We generally get along great, but nobody's going to be as brutally honest as your own family members. So we tend to kind of push eachother to the edge, so to speak. And in a weird way, it's definitely helped me to realize that nobody's going to stand up for you. At times, you just have to stand up and give it your all.
Anyway, what I've learned through my 18 year existence so far is that emotions play a huge role in the shy/unassertive process. If you keep stuffing your emotions down, you're going to get hurt. And the more you get hurt is the worse it's going to be when the cap finally comes off. I used to cry (cause I'm a pussy), but now I just get angry. And somehow, I'd rather have that than to keep stuffing my true feelings down inside.
So my tip - if you can, let her know that she's awesome and she should never let anyone step on her. Sounds like you really care about her, so that always helps when you're trying to get the point across. Also, let her know that she should never be taken advantage of. It's human nature to dominate those that are weak, sort of like what dogs and wolves do - so if somebody perceives her as weak (example, your younger relative), that person will not hesitate to do whatever they want, because they know she won't speak up. However, is she does speak up, they'll more than likely leave her alone.
Like I said, I've been walked on many times due to this mentality by friends, family members, a dickwad of a boss - and I've finally realized why the hell people always seem to be fine with fucking with me.. or just being as brutally honest as they please - because they know I won't say anything in return. I guess you just gotta let that cap off and spray your feelings (weather it's anger, sadness, excitement, happiness etc) out and let it be known that you're not gonna take shit. Somehow it actually works