Where to stay for a weekend visit?

We are going to Charleston for the first time on a Friday-Monday trip. Does it matter where we stay? Our goal is to explore the city but we aren’t really into museums or anything like that just seeing the city itself. Trying to find a good location for an air b and b. Should we stay in the historic district? Or the French quarter? Or somewhere else and just drive everywhere?