Options -Indexes in apache does not work, and any edits display a 500 error

I have seen this asked multiple times, but none are answering my issue (using apache). What I'm trying to do is simply disable all directory listing, and showing a 403. People/servers will still have access to the files knowing the path, but will not be able to browse and will instead see a 403 forbidden. I only want to apply this to specific directories, not all of them in the root. I've been trying to implement .htaccess files in these directories, but none of them have effect. Allow Override All or Allow Override None simply causes 500 code errors, and leaving the .htaccess in the subdirectory with "Options -Indexes" has no effect. What am I doing wrong here? Should I be putting something in the main .htaccess that I'm not?