What does lack of confidence mean?

My self-confidence as a philosopher seems to be, not a barometer for whether I should continue on in philosophy, but an indirect measure of who is in the room.  I had the same experience as the author in “The Stereotype Threat Room:” in a room of mostly white men, my confidence, concentration, and performance for a familiar talk (given successfully elsewhere) plummeted.  When I attend talks, my confidence and concentration also take a hit when the audience is overwhelmingly white and male.  As I continue on in philosophy, it feels disorienting and inauthentic to distrust my own feelings of self-confidence.  But, understanding why these feelings can be so volatile helps a great deal.