If I download stuff from the internet does it affect the internet connection of all the computers in our house?

[just me]

Sooo I have my own laptop in my room and my brother always uses the one we have for the whole family,we both use the same internet.

The thing is that my brother always starts complaining and accusing me because the internet is slow. He's always like "are you downloading something ? the internet is so slow !! "..i just don't get it,because even when i'm not downloading anything he still complains to me how the internet is laggy. If he downloads something it doesn't affect the internet for me..I think that it really doesn't matter if you're downloading something or not,the internet connection should be about the same,it shouldn't affect it too much,am i totally wrong?..

I'm getting tired of him always blaming me for the same thing over and over again,i never complain to him because i don't see a reason to.How can i tell him that it's not all about me downloading stuff,it could be other stuff that affects the internet connection as well?..