You're going about it all wrong just like most parents. To you, it's all about how her behaviour effects YOU. Sure she's a brat but WHY is she a brat? People think kids act like this just to piss you off and they don't. Get to the bottom of the real issue. Try watching SuperNanny or something where they deal with kids throwing temper tantrums.
Your brother is right on. "she is unhappy with life" and you cannot punish her and take away things for being unhappy. 2 negatives don't make a positive. Her situation sounds like it sucks. Look at it from her perspective. So she has a mother 50% of the time, You for 30% of the time and her father for 20% of the time? How does she live a 50/50 split life? So 3 days a week she sleeps over your house and 4 days she sleeps over her mom's house? Yeah, that sucks. Who the hell wants to live somewhere and sleep in a different bed half the week and somewhere else the rest? I'd be miserable too. Usually in joint custody the kid just does a sleepover with the father on the weekend and a visit during the week when he has time; not live half and half and then be babysat by the replacement.(no offense, but from a kid's point of view that's what you are) This kid sounds like she's being bounced back and forth rather than being a normal kid. Kids need routine and security.
How much attention does she get from her father? He comes home at 6pm and THEN she has to go to bed an hour later? Obviously not much time at all. And what's with homework cutting into special time anyway? WHY oh WHY does she have to WORK to get her father's attention? Do homework, clean room and her reward is some special time with daddy? What is that? Kids shouldn't have to EARN their parent's love and attention.
She probably feels that her mom and dad dumped her on you and then dad coming home for an hour to dump her into bed so he can spend the rest of his time with you. If all he has is an hour to spend with her a day why is she spending 50% of her time there? If his work schedule is that bad she should be home with her mother; not with you. At the least you could let her stay up an hour later but then it cuts down on YOUR time and you'll have another problem.
Do you ever see a pattern in her behaviours or what sets her off? Look for one. Take notes and keep a journal. Like does her Daddy say something nice to your son so she acts up to get his attention back on her? Does she act up when he comes home late?
She is treating your son like dirt probably because she's jealous. Your son has HER dad full time and her father replaced her with your son. He was her dad for 6? years untill you came along and now she has to share him with you and your son. She might do potty talk with your son because it's getting a rise out of you OR because it makes her feel good that your son is laughing at her and thinks she's funny.
She's acting out during those 2.5 hrs because for some reason she needs attention or she is just miserable with her situation. Why do you pick her up from school anyway? Do you live in a different city? Does she take the bus home at her mom's house? If so, one issue with you picking her up is that she could be missing fun time she enjoys with her friends on the bus.
Long day at school and now she has to go home with you and do more homework and do chores? Does she have much fun time ? What is her life like over her mom's house? Mom spend time with her or dump her on a bf/husband/babysitter?
Do you guys ever do much as a family? Have you tried to see past the brattiness and get to know her? And without your son around? It's fine that you don't instantly "Love" her but you need to try and bond with her and form a relationship. Primary CareGiver doesn't have to mean EvilStepMom.
What is your or your husband's relationship like with her mother? If you can talk to the mother and the 3 of you can go for help that would be your best bet. Like you said, hard to change when the other half is reinforcing it by babying her and probably spoiling her.
My suggestions -
Suggest that her parents come up with a new custody plan rather than bouncing her around and giving her 2 seperate lives. Her time with her father would be more benefical if she spent a few hours with him doing something special once or twice a week when he actually has the time.
Try to bond with her. Leave your son with a babysitter one day and pick her up by yourself and suprise her by asking if there's anything she'd like to go do.
Get her in a good mood and try talking to her and finding out what's really upsetting her. Ignore the brattiness. Don't talk down to her. Ask her questions like does she enjoy coming over your house, does she like you picking her up from school, if there's anything she'd rather do. etc.
Try family counseling with you, her mother and her father.
Tell your husband it's easier for him to accuse you because he's not there most of the time when she's at her worst and dealing with what you're dealing with. Tell him you dont want to fight with him about it and don't want it to be the break in your marriage and you'd like to try family counseling so you can work together to find a solution instead of arguing about it. Get him to agree with you that obviously there is a serious problem getting worse if doors are coming off hinges and things are breaking and patience isn't going to make it stop.