is it possible with out messing up my account to play my xbox one at my home and my brother play my xbox 360 at his home on my account will it ruin my account if we do so.
Yes and no. Your microsoft account can be signed in to both a Xbox 360 and a Xbox One at the same time, without any issues.
As a secondary issue you face, it is against the terms of service to share your xbox account with another person. While I am personally doubtful that Microsoft would act against you, you should be aware that using differant WiFi connections suggests access from differant IP addresses, which in turn could raise a red flag.