Each field we add slows the search slightly so we try to keep it to key fields, I was going to suggest searching by email address but it looks like you've already got that covered. You could add a sub filter to the email field if you want to tighten this up further when you get too many results.