Getting rid of file extensions such as .html and .php

There is no other way than modrewrite that I'm aware of, so yes you have to use a htaccess file.

Some tips for beginners: http://corz.org/serv/tricks/htaccess2.php

Documentation: http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html