I promise that I did search for the forums for about an hour before posting this 
Despite that effort, I just found the solution here while browsing some more.
The slim+skim I was using already made a selection so that a collection like FatJet would not be empty meaning my code worked fine. However, the central NanoAOD does not have that protection and as explained in the post, the RDataFrame Filter will not do that check for you!