The simplest and most direct way to permanently delete files or folders is by using the Shift + Delete keyboard shortcut.