I know for a fact you don't have to do the mini dungeons because on my alt Basswhole she(weird calling your alt a she lol) I was doing all of them and I have not done one quest on her or a mini thing. As for shadow caves make sure you finish them all and get a good group!
Bookmarks