Yes. Responding quickly to LSA leads can affect account performance because Google considers responsiveness when evaluating local service providers. Fast responses can also improve your chances of turning a lead into a booked job before the customer contacts another business.
For calls, try to answer during your listed business hours. For missed calls, return them quickly, and respond to messages as soon as possible. A slow response does not automatically cause a suspension or ranking drop, but repeated missed calls and delayed replies can reduce your ability to convert leads and may weaken overall performance.
Track response time, missed calls, returned calls, message response time, qualified leads, and booked jobs. The goal is not simply to respond quickly, but to respond quickly to the right leads and turn them into customers.