There is a time and a place for doing that; But that time is not all the time and that place is not everywhere.
Your personal and business circles should not consist of people you need to “dumb it down” for so they can feel in rapport with you, or so they can understand you, or so that you feel you fit in with them. Fitting in by definition makes you average.
Either find a group you don’t need to “dumb down” to hang with (ideally, one that you have to step it up to keep up with), or start your own group — this group may be only one person deep for some time.
If that makes you uncomfortable, you always have the option of fitting in.