If you knew how to install Wordpress all on your own, you're above average. If you figured out how to upgrade to the latest version (up to 2.3.3 now), you're probably better than most when it comes to your technical skills. The long story short, most Wordpress users don't upgrade their installations once they are up and running. Basically, they think that if everything is working ok, what's the point?
Well, everything is working ok for now.
Eventually, your site can/will be hacked. I can't tell you how many new client's I've "earned" because they needed someone to restore their messed up Wordpress install that had been hacked.
Think about it, as soon as a vulnerability has been detected and Wordpress releases a new version, the entire world is alerted to the security hole. So how long do you think it will be before your blog is discovered with that big, gaping hole?
Well, one of the things you need to be aware when upgrading is the need to do a complete upgrade. You really should re-upload all fresh files.
As ShoeMoney points out, not only is it a good idea to install fresh system files, but you should also change your system password. You never know who might already have hacked your blog. If they've got the password, it won't matter how often you upgrade.