I put SP3 on both my wife's and my computer.  No problems during installation (took about 45 min) but no great differences in performance afterwords either.

SP3 supposedly incorporates all of the critical fixes that MS has put out so it should be a more secure operating system.

If you don't install it and somewhere down the line you pick up a nasty virus you may wonder if it got in through one of the security holes that SP3 would have fixed.

If you've got the time I'd upgrade. Another reason to upgrade is that as time goes by more and more software may require SP3 to install.