My Linux PC has been hacked once and attempted a second hack. What do I do?

[masteroftitanic]

Last night, on Facebook, me and my friend were discussing, things, that could'v sparked interest to like the government or something. We were talking about all the 9/11 conspiracy theories, we also talked about how we thought where we live might be a terrorist target. I mentioned we should move to Canada (nobody hates Canada). Then my jackass friend sent the message "and bomb it". Right after that I got a notification that Remote Desktop had been activated. My security settings were modified so they didn't need my password or my confirmation to access my computer. I was unable to disconnect them so I instantly unplugged my computer. I logged back in a few hours later. i fixed my security settings, checked facebook and talked to the jackass friend that started it. He told me that his computer was also accessed, but he uses Windows. After a couple minutes of talking to him we both went to bed. I woke up this morning with a message on my computer screen. They tried to connect again while i was sleeping. I got their IP address.

Their IP address was:
static.121.187.9.5.clients.your-server.de

I tried looking up this IP and came to a website, a Japanese website, this IP was mixed in with a bunch of other ones.
What should I do? Should I be worried?

This is my computers OS:
Linux Ubuntu 11.04 Natty Narwhal
Stupid Yahoo cut off the end of the IP address above

static. 121. 187 . 9 .5 .clients. your- server. de
Okay I just found out its not ONE IP address is trying to access me, theres many. I just got ANOTHER notification of someone trying to get into my computer but now theres a different IP.
Now more and more are coming. It's not the US Government as far as I know. I tracked their IP addresses and these computers are trying to access me from GERMANY AND RUSSIA.
One after another from all corners of the freaking world they wont stop!!!!