Can I delete a folder?

Yes, you can, however when you delete a folder you also delete everything stored in that folder. You click on the folder icon (not the name) and the details of the folder will come up on the right-hand side. From there you select the delete button.