Before you delete a task, consider completing it instead. This will remove it from your task list after a day, but Outlook will keep the completed task, and you can reopen it from Outlook or from the app, as described in Reopening completed tasks. For example, if you have a root task called Client 1, and you complete all your projects for the client, you should consider completing the task rather than deleting it, so that you don't have to create a new task when the client gives you another project.
If you want to delete a task, right-click on it in the task list, and select "Delete task." This will delete the selected task, all its child tasks, and the copies that Tasklist made of linked emails and notes. However, it won't delete linked emails and notes from Outlook - just the copies that Tasklist made.
Deleting a task in Outlook does not delete it from the list in the app. To delete a task from the list in the app, right-click on it in the app, and select "Delete task." If you just delete the task in Outlook, then double-click on the task in the app, the app will create a new Outlook task.