Yes, Yale expects you to send them ALL of your test scores. But your one “bad” score will not affect your chances whatsoever. Yale will officially “use” only two scores. Sure, the admission folks may see the other two, but given that you did so extremely well on three tests, the lower score will mean absolutely nothing.
Trust “The Dean” … if you don’t get into Yale, it won’t be because of that 640. The hyper-competitive colleges turn away thousands of fully qualified students every year who would thrive there if accepted.