Fauci asked if he thinks he 'doing more harm than good at this point'