I was creating a number of user stories and associating the Tasks to users, and quickly realized that the TFS 2013 still does not have a way to delete a task.  Woooaahhh!!!

There is a way to do it via command line, via witadmin tool:

witadmin destroywi /Collection:<TFS Collection URL> /id:<id of the task>